US 11,057,567 B2 |
||

Anti-shake method and apparatus for panoramic video, and portable terminal |
||

Cong Chen, Shenzhen (CN); and Jinlin Cai, Shenzhen (CN) |
||

Assigned to ARASHI VISION INC., Shenzhen (CN) |
||

Appl. No. 16/759,315 |
||

Filed by ARASHI VISION INC., Shenzhen (CN) |
||

PCT Filed Oct. 16, 2018, PCT No. PCT/CN2018/110467§ 371(c)(1), (2) Date Apr. 24, 2020, PCT Pub. No. WO2019/080748, PCT Pub. Date May 2, 2019. |
||

Claims priority of application No. 201711015180.0 (CN), filed on Oct. 25, 2017. |
||

Prior Publication US 2020/0314340 A1, Oct. 1, 2020 |
||

This patent is subject to a terminal disclaimer. |
||

Int. Cl. (2006.01); H04N 5/232 (2006.01); G01P 3/22 (2013.01)G01P 15/18 |

CPC (2013.01) [H04N 5/23258 (2013.01); G01P 3/22 (2013.01); G01P 15/18 (2013.01)]H04N 5/23238 |
18 Claims |

1. An anti-shake method for a panoramic video, comprising steps of:
acquiring, in real time, a current state timestamp, an accelerometer data and an angular velocity of a gyroscope in a portable terminal;
estimating a rotation from the portable terminal to a world coordinate system using Extended Kalman Filtering combined with both the accelerometer data and the angular velocity;
synchronizing the timestamp of the gyroscope with a timestamp of a panoramic video frame;
performing quaternion interpolation in a rotation from the portable terminal to a world coordinate system to obtain a rotation matrix corresponding to the panoramic video frame; and
rotating a panoramic image according to the current rotation matrix to generate a stable video frame;
wherein the step of estimating a rotation from the portable terminal to a world coordinate system using Extended Kalman Filtering combined with both the accelerometer data and the angular velocity, specifically comprises steps of:
S
1021, calculating an initial-state rotation q_{0}^{+} and an initial process covariance matrixwhere, d
_{0 }is a measured initial accelerometer data, g is the gravity vector of the world coordinate system; an initial process covariance matrixS
1022, calculating a State Transition Matrix Φ(ω_{k}) at time K using the angular velocity ω_{k};Φ(ω
_{k})=exp(−[ω_{k}·Δt]_{x}), where, ω_{k }is an angular velocity at time K, Δt represents a sampling time interval of the gyroscope;S
1023, calculating a State Noise Covariance Matrix Q_{k}, updating an a priori estimate for a state rotation q_{k}^{−} and an a priori estimate for a process covariance matrix p_{k}^{−};where Q
_{k }is a State Noise Covariance Matrix;q
_{k}^{−}=Φ(ω_{k})*q_{k−1}^{+}, where, q_{k−1}^{+}, is an a posteriori estimation for a state rotation at time K−1;P
_{k}^{−}=q_{k}^{−}·P_{k−1}^{+}·q_{k}^{−}+Q_{k}, where, P_{k−1}^{+} is an a posteriori estimation for a process covariance matrix at time K−1;S
1024, updating an Observation Noise Covariance Matrix V_{k }from the accelerometer data d_{k}, calculating an Observation Transformation Jacobian Matrix H_{k}, and calculating an Observation Error e_{k }between the current observation and estimated observation;where, δ
_{k}=δ_{k}^{var}+δ_{k}^{mean}, δ_{k}^{var}=α∥d_{k}|−|d_{k−1}∥+(1−α)δ_{k−1}^{var},α is a smoothing factor for an acceleration change, and β is an impact factor for an acceleration module;
where, h is an observation function, h(q,v)=q·g+I
_{k}, g is the gravity vector of the world coordinate system, q is a state rotation, that is, a rotation from the world coordinate system to the gyroscope coordinate system; I_{k }is a measurement noise;e_{k}=d_{k}×h(q_{k}^{−},0);S
1025, updating an optimal Kalman Gain Matrix K_{k }at time k;K_{k}=P_{k}^{−}·H_{k}^{T}(H_{k}·P_{k}^{−}·H_{k}^{T}+V_{k})^{−1}; andS
1026, updating the a posteriori estimation for a rotation q_{k}^{+ }from the portable terminal to the world coordinate system, and the a posteriori estimation for the process covariance matrix p_{k}^{+ }according to the optimal Kalman Gain Matrix K_{k }and the Observation Error e_{k}; |