US 12,002,168 B2
Low latency hand-tracking in augmented reality systems
Jan Bajana, Vienna (AT); Bernhard Jung, Perchtoldsdorf (AT); and Daniel Wagner, Vienna (AT)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Jun. 20, 2022, as Appl. No. 17/844,541.
Prior Publication US 2024/0096026 A1, Mar. 21, 2024
Int. Cl. G06T 19/00 (2011.01); G06T 7/20 (2017.01); G06T 15/00 (2011.01)
CPC G06T 19/006 (2013.01) [G06T 7/20 (2013.01); G06T 15/005 (2013.01); G06T 2200/28 (2013.01); G06T 2207/30196 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
accessing a first frame from a camera of an Augmented Reality (AR) device;
tracking, using a computer vision engine of the AR device, a first image of a hand in the first frame;
rendering, using a render engine of a Graphical Processing Unit (GPU) of the AR device, virtual content based on the tracking of the first image of the hand in the first frame, wherein the camera and the render engine operate at a same frequency;
accessing a second frame from the camera before the rendering of the virtual content is completed, the second frame immediately following the first frame;
tracking, using the computer vision engine of the AR device, a second image of the hand in the second frame;
generating an annotation based on tracking the second image of the hand in the second frame;
forming an annotated virtual content based on the annotation and the virtual content; and
displaying the annotated virtual content in a display of the AR device.