US 12,067,693 B2
Late warping to minimize latency of moving objects
Bernhard Jung, Perchtoldsdorf (AT); and Daniel Wagner, Vienna (AT)
Assigned to SNAP INC., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Nov. 4, 2021, as Appl. No. 17/518,828.
Claims priority of provisional application 63/190,119, filed on May 18, 2021.
Prior Publication US 2022/0375026 A1, Nov. 24, 2022
Int. Cl. G06T 3/18 (2024.01); G06T 13/40 (2011.01); G06T 13/80 (2011.01); G06T 15/20 (2011.01)
CPC G06T 3/18 (2024.01) [G06T 13/40 (2013.01); G06T 13/80 (2013.01); G06T 15/205 (2013.01); G06T 2210/44 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for minimizing latency of moving objects comprising:
determining an initial pose of a visual tracking device of an augmented reality device that comprises an optical sensor;
accessing an image generated by the optical sensor;
identifying an initial location of an object depicted in the image, the image corresponding to the initial pose of the visual tracking device, the object comprising a first physical object that is moving and a second physical object that is stationary;
rendering a first layer of virtual content based on the initial pose of the visual tracking device and a first initial location of the first physical object;
rendering a second layer of virtual content based on the initial pose of the visual tracking device and a second initial location of the second physical object;
retrieving an updated pose of the visual tracking device;
detecting and tracking, using computer vision, an updated location of the first physical object in an updated image that corresponds to the updated pose; and
applying a time warp transformation to the rendered virtual content based on the updated pose and the updated location of the object to generate transformed virtual content by applying a first time warp transformation to the first layer based on the updated pose and the updated location of the first physical object, applying a second time warp transformation to the second layer based only on the updated pose, and combining the first layer and the second layer in a single rendered frame.