US 11,854,227 B2
Depth-from-stereo bending correction using visual inertial odometry features
Sagi Katz, Yokneam Ilit (IL); and Matthias Kalkgruber, Vienna (AT)
Assigned to SNAP INC., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Sep. 21, 2021, as Appl. No. 17/480,405.
Claims priority of provisional application 63/188,815, filed on May 14, 2021.
Prior Publication US 2022/0366600 A1, Nov. 17, 2022
Int. Cl. G06T 7/33 (2017.01); G06T 7/73 (2017.01); G06T 7/593 (2017.01); G06T 19/00 (2011.01); H04N 13/239 (2018.01)
CPC G06T 7/74 (2017.01) [G06T 7/33 (2017.01); G06T 7/593 (2017.01); G06T 19/006 (2013.01); G06T 2207/10012 (2013.01); H04N 13/239 (2018.05)] 20 Claims
OG exemplary drawing
 
1. A method for correcting a bending of a flexible device comprising:
accessing feature data of a first stereo frame that is generated by stereo optical sensors of the flexible device, the feature data generated based on a visual-inertial odometry (VIO) system of the flexible device;
accessing depth map data of the first stereo frame, the depth map data generated based on a depth map system of the flexible device;
estimating a pitch-roll bias based on the feature data and the depth map data of the first stereo frame by: projecting stereo VIO features of the first stereo frame to a two-dimensional coordinate system, and computing the pitch-roll bias based on a misalignment in the projected stereo VIO features in the two-dimensional coordinate system; and
generating a second stereo frame after the first stereo frame, the second stereo frame based on the pitch-roll bias of the first stereo frame.