US 12,136,177 B2
Machine vision hand and tool tracking
Jon Danielson, Fort Collins, CO (US); Adolfo Adolfo Gomez, Fort Collins, CO (US); and Mark Lessman, Fort Collins, CO (US)
Assigned to Hewlett-Packard Development Company, L.P., Spring, TX (US)
Filed by HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., Spring, TX (US)
Filed on Oct. 21, 2022, as Appl. No. 17/970,849.
Prior Publication US 2024/0135653 A1, Apr. 25, 2024
Prior Publication US 2024/0233278 A9, Jul. 11, 2024
Int. Cl. G06T 19/00 (2011.01); G06F 3/01 (2006.01); G06T 7/246 (2017.01); G06T 7/73 (2017.01); G06V 20/20 (2022.01)
CPC G06T 19/006 (2013.01) [G06F 3/011 (2013.01); G06T 7/246 (2017.01); G06T 7/73 (2017.01); G06V 20/20 (2022.01); G06T 2207/30196 (2013.01)] 20 Claims
OG exemplary drawing
 
1. An extended reality (XR) system, comprising:
an imaging system to present virtual objects to a user;
a machine vision tracking system comprising:
a camera to capture images of the user's hand and a tool grasped in the user's hand; and
a processor to:
identify the tool grasped in the user's hand;
differentiate the tool grasped in the user's hand from the user's hand by:
extracting, from the images, a set of registration features of the user's hand that define a position and orientation of the user's hand; and
extracting, from the images, a set of registration features of the tool that define a position and orientation of the tool;
track the position and orientation of the user's hand in physical space; and
track the position and orientation of the tool in physical space; and
an XR controller to manipulate an XR environment based on:
the position and orientation of the tool; and
the position and orientation of the user's hand.