US 12,112,418 B2
Volumetric capture and mesh-tracking based machine learning 4D face/body deformation training
Kenji Tashiro, San Jose, CA (US); and Qing Zhang, San Jose, CA (US)
Assigned to SONY GROUP CORPORATION, Tokyo (JP)
Filed by SONY GROUP CORPORATION, Tokyo (JP)
Filed on Mar. 23, 2023, as Appl. No. 18/125,659.
Application 18/125,659 is a continuation of application No. 17/219,698, filed on Mar. 31, 2021, granted, now 11,640,687.
Claims priority of provisional application 63/003,097, filed on Mar. 31, 2020.
Prior Publication US 2023/0230304 A1, Jul. 20, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 13/40 (2011.01); G06N 20/00 (2019.01); G06T 17/20 (2006.01); G06V 40/10 (2022.01); G06V 40/20 (2022.01)
CPC G06T 13/40 (2013.01) [G06N 20/00 (2019.01); G06T 17/20 (2013.01); G06V 40/103 (2022.01); G06V 40/23 (2022.01)] 15 Claims
OG exemplary drawing
 
1. A method programmed in a non-transitory memory of a device comprising:
using a volumetric capture system for high-quality 4D scanning, wherein the volumetric capture system is configured for capturing high quality photos and video simultaneously;
using mesh-tracking to establish temporal correspondences across a 4D scanned human face and full-body mesh sequence;
using mesh registration to establish spatial correspondences between the 4D scanned human face and full-body mesh sequence and a 3D computer graphics physical simulator; and
training surface deformation as a delta from the 3D computer graphics physical simulator using machine learning including combining detail-rich implicit functions and parametric representations to construct 3D models.