US 12,272,001 B2
Rapid generation of 3D heads with natural language
Joseph Logan Olson, San Mateo, CA (US); Mager Kamel Aquino, Montreal (CA); and Jade Raymond, Montreal (CA)
Assigned to Sony Interactive Entertainment LLC, San Mateo, CA (US)
Filed by Sony Interactive Entertainment LLC, San Mateo, CA (US)
Filed on Sep. 30, 2022, as Appl. No. 17/937,418.
Prior Publication US 2024/0112403 A1, Apr. 4, 2024
Int. Cl. G06T 17/20 (2006.01); G06F 40/40 (2020.01); G06N 3/08 (2023.01)
CPC G06T 17/20 (2013.01) [G06F 40/40 (2020.01); G06N 3/08 (2013.01)] 8 Claims
OG exemplary drawing
 
1. A device comprising:
at least one computer storage that is not a transitory signal and that comprises instructions executable by at least one processor system to:
generate a base three dimensional (3D) neural radiance field (NeRF) from plural images;
use text input to a Contrastive Language-Image Pre-training (CLIP) model to generate a modified NeRF from the base 3D NeRF; and
convert the modified NeRF to a polygonal mesh representing a virtual human head for presentation of the virtual human head in at least one computer simulation; wherein the instructions are executable to:
use a machine learning (ML) model on the base 3D NeRF to minimize a loss indication in matching the text;
train the ML model on a chain of causality from initial image parameters that control vertices of an object to pixels of the object rendered onscreen.