US 11,865,444 B2
Tracking hand gestures for interactive game control in augmented reality
Ilteris Canberk, Marina Del Rey, CA (US); Dmytro Kucher, Odessa (UA); Dmytro Hovorov, Luhansk (UA); and Artur Sydoran, Uzhhorod (UA)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Ilteris Canberk, Marina Del Rey, CA (US); Dmytro Kucher, Odessa (UA); Dmytro Hovorov, Luhansk (UA); and Artur Sydoran, Uzhhorod (UA)
Filed on Sep. 18, 2020, as Appl. No. 17/024,800.
Prior Publication US 2022/0088476 A1, Mar. 24, 2022
Int. Cl. A63F 13/428 (2014.01); A63F 13/213 (2014.01); A63F 13/26 (2014.01); A63F 13/537 (2014.01); G06F 3/01 (2006.01); G06F 3/042 (2006.01); G06F 3/04817 (2022.01); G06T 11/60 (2006.01)
CPC A63F 13/428 (2014.09) [A63F 13/213 (2014.09); A63F 13/26 (2014.09); A63F 13/537 (2014.09); G06F 3/017 (2013.01); G06F 3/0425 (2013.01); G06F 3/04817 (2013.01); G06T 11/60 (2013.01); A63F 2300/8082 (2013.01); G06T 2210/12 (2013.01); G06T 2210/21 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method of presenting an interactive game in augmented reality, said method comprising:
capturing frames of video data with a camera, wherein each frame of video data comprises depth information for a plurality of pixels, and wherein said camera is coupled to an eyewear device comprising a processor, a memory, a hand tracking utility, and a display;
presenting a game clock on said display at a location establishing a field of play, wherein said field of play is persistently viewable as an overlay relative to a physical environment, independent of movement of said eyewear device, and wherein said game clock is configured to increment from a start time to a final time;
detecting in said frames of video data, with said hand tracking utility, a final hand shape associated with said final time;
identifying a set of hand landmarks associated with said detected final hand shape, based on said depth information;
selecting a candidate hand gesture from a library of hand gestures based on said identified set of hand landmarks;
selecting a final virtual game piece from a set of virtual game pieces based on said selected candidate hand gesture;
presenting on said display said final virtual game piece as an overlay relative to said final hand shape;
broadcasting wirelessly said final virtual game piece;
detecting an opposing final virtual game piece; and
presenting on said display a game score based on said final virtual game piece versus said opposing final virtual game piece.