US 12,437,470 B1
Neural shading
Graham L. Fyffe, Culver City, CA (US); and Andrew P. Mason, Cottesloe (AU)
Assigned to Apple Inc., Cupertino, CA (US)
Filed by Graham L. Fyffe, Culver City, CA (US); and Andrew P. Mason, Cottesloe (AU)
Filed on Jul. 7, 2022, as Appl. No. 17/811,130.
Claims priority of provisional application 63/219,171, filed on Jul. 7, 2021.
Int. Cl. G06T 15/80 (2011.01); G06T 7/40 (2017.01); G06T 15/04 (2011.01)
CPC G06T 15/80 (2013.01) [G06T 7/40 (2013.01); G06T 15/04 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A non-transitory computer-readable medium comprising computer-readable code executable by one or more processors to:
initiate an image-based communication session between a local device and a remote device;
obtain one or more identity textures for a user, wherein the one or more identity textures are generated prior to the image-based communication session; and
for each of one or more frames of the communication session:
receive, from the remote device, shading latents, wherein the shading latents comprise an encoded set of concatenated latents comprising mesh values, lighting values, and identity values for the user,
apply the shading latents to a decoder to obtain one or more neural maps, and
generate a target texture by warping the one or more identity textures in accordance with the one or more neural maps for a particular frame of the one or more frames.