US 12,347,011 B2
Generating physio-realistic avatars for training non-contact models to recover physiological characteristics
Daniel J. McDuff, Cambridge, MA (US); Javier Hernandez Rivera, Cambridge, MA (US); Tadas Baltrusaitis, Cambridge (GB); and Erroll William Wood, Cambridge (GB)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Sep. 11, 2023, as Appl. No. 18/244,622.
Application 18/244,622 is a continuation of application No. 16/907,110, filed on Jun. 19, 2020, granted, now 11,790,586.
Prior Publication US 2023/0419581 A1, Dec. 28, 2023
Int. Cl. G06T 13/40 (2011.01); G06N 5/04 (2023.01); G06N 20/00 (2019.01); G06V 20/40 (2022.01)
CPC G06T 13/40 (2013.01) [G06N 20/00 (2019.01); G06V 20/46 (2022.01); G06F 2218/00 (2023.01)] 20 Claims
OG exemplary drawing
 
1. A method for generating video sequences including physio-realistic avatars, the method comprising:
receiving an albedo for an avatar;
modifying a sub-surface skin color associated with the skin pixels of the albedo based on a sub-surface skin color weight applied to physiological data, wherein the sub-surface skin color weight is based on an absorption spectrum associated with the physiological data for one or more color channels;
generating physio-realistic avatar video of the avatar by rendering the avatar based on the albedo and the modified sub-surface skin color;
generating, based on the generated physio-realistic avatar video, a machine learning structure, wherein the machine learning structure comprises a first model and a second model, the first model predicts a motion representation of the avatar, and the second model predicts a region in a frame of the generated physio-realistic avatar video for computing a physiological signal; and
storing the generated machine learning structure and one or more weights of the generated machine learning structure, wherein the one or more weights of the generated machine learning structure represent one or more model weights of a trained machine learning model.