US 11,792,517 B2
Pose tracking for rolling shutter camera
Georg Halmetschlager-Funek, Vienna (AT); Jeroen Hol, Hengelo (NL); Matthias Kalkgruber, Vienna (AT); Nikolaj Kuntner, Vienna (AT); and Daniel Wolf, Modling (AT)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Georg Halmetschlager-Funek, Vienna (AT); Jeroen Hol, Hengelo (NL); Matthias Kalkgruber, Vienna (AT); Nikolaj Kuntner, Vienna (AT); and Daniel Wolf, Modling (AT)
Filed on Dec. 28, 2020, as Appl. No. 17/135,055.
Claims priority of provisional application 63/085,775, filed on Sep. 30, 2020.
Prior Publication US 2022/0103734 A1, Mar. 31, 2022
Int. Cl. H04N 23/68 (2023.01); G06T 7/73 (2017.01); H04N 25/531 (2023.01)
CPC H04N 23/689 (2023.01) [G06T 7/73 (2017.01); H04N 25/531 (2023.01); G06T 2207/30244 (2013.01)] 21 Claims
OG exemplary drawing
 
1. A method for tracking poses of a rolling shutter camera in an augmented reality system, comprising:
capturing an image using the rolling shutter camera, the image comprising a plurality of image scan lines;
estimating a reference pose of the rolling shutter camera at a reference line, the reference line being one of the plurality of the image scan lines;
estimating relative poses of the rolling shutter camera to the reference pose of the rolling shutter camera using inertial sensor data, each relative pose of the rolling shutter camera corresponding to an image scan line other than the reference line, wherein the corresponding relative pose of each image scan line is estimated by concatenating all pose changes between the reference pose and the corresponding relative pose being estimated; and
refining the reference pose of the rolling shutter camera based on the relative poses of the rolling shutter camera in combination with information contained in the image.