US 12,314,485 B2
Device-to-device collocated AR using hand tracking
Jordy Innocentius Ajanohoun, Vienna (AT); Markus Diem, Vienna (AT); Georgios Evangelidis, Vienna (AT); and Matthew Penney, Vienna (AT)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Jul. 21, 2023, as Appl. No. 18/357,050.
Claims priority of application No. 20230100308 (GR), filed on Apr. 11, 2023.
Prior Publication US 2024/0345665 A1, Oct. 17, 2024
Int. Cl. G06F 3/01 (2006.01); G06F 3/0346 (2013.01); G06T 7/70 (2017.01); G06T 7/80 (2017.01)
CPC G06F 3/0346 (2013.01) [G06F 3/011 (2013.01); G06T 7/70 (2017.01); G06T 7/80 (2017.01); G06T 2207/30196 (2013.01); G06T 2207/30244 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A method comprising:
accessing first pose data of a first handheld device;
receiving second pose data of a second handheld device;
detecting, from the first handheld device, hand-tracking data of a second user holding the second handheld device; and
aligning a first coordinate system of the first handheld device with a second coordinate system of the second handheld device based on the first pose data, the second pose data, and the hand-tracking data of the second user holding the second handheld device,
wherein detecting, from the first handheld device, the hand-tracking data of the second user holding the second handheld device comprises:
capturing, using a first camera of the first handheld device, an image of one or more fingers of a hand of the second user;
identifying, using a hand-tracking recognition process, a position of a hand joint based on the image of the one or more fingers of the hand of the second user;
identifying a first coordinate transformation between the position of the hand joint and a second pose of the second handheld device, the second pose based on the second pose data; and
identifying a second coordinate transformation between the position of the hand joint and a first pose of the first handheld device, the first pose based on the first pose data.