US 11,887,267 B2
Generating and modifying representations of hands in an artificial reality environment
Christopher James Goodman, Round Rock, TX (US); and Gregory Mayo Daly, Seattle, WA (US)
Assigned to Meta Platforms Technologies, LLC, Menlo Park, CA (US)
Filed by Meta Platforms Technologies, LLC, Menlo Park, CA (US)
Filed on Jul. 7, 2021, as Appl. No. 17/369,354.
Prior Publication US 2023/0009367 A1, Jan. 12, 2023
Int. Cl. G06T 19/20 (2011.01); G06T 15/04 (2011.01); G06T 7/73 (2017.01); G06T 15/50 (2011.01)
CPC G06T 19/20 (2013.01) [G06T 7/75 (2017.01); G06T 15/04 (2013.01); G06T 15/506 (2013.01); G06T 2200/08 (2013.01); G06T 2207/30196 (2013.01); G06T 2210/12 (2013.01); G06T 2210/62 (2013.01); G06T 2219/2004 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A method comprising, by a computing system:
receiving an image of a real environment captured using a camera worn by a user, the image comprising a hand of the user;
determining a pose of the hand based on the image and a light direction of a light source;
generating, based on a three-dimensional model of the hand having the determined pose, a first two-dimensional surface representing the hand as viewed from a first viewpoint of the user;
projecting, according to the light direction, the three-dimensional model of the hand having the determined pose onto one or more virtual objects;
generating a second two-dimensional surface representing a shadow of the hand on the one or more virtual objects as viewed from the first viewpoint of the user;
positioning the first two-dimensional surface representing the hand, the second two-dimensional surface representing the shadow of the hand on the one or more virtual objects, and one or more virtual-object representations of the one or more virtual objects in a three-dimensional space;
determining that a first portion of the first two-dimensional surface representing the hand and a second portion of the second two-dimensional surface representing the shadow of the hand are visible from a second viewpoint in the three-dimensional space; and
generating an output image, wherein (1) a first set of image pixels of the output image corresponding to the portion of the first portion of the first two-dimensional surface that is visible is configured to cause a display to turn off a set of corresponding display pixels, and (2) a second set of image pixels of the output image is generated using the second portion of the second two-dimensional surface and one or more associated transparency levels.