US 12,437,202 B2
Human characteristic normalization with an autoencoder
Javier Hernandez Rivera, Boston, MA (US); Daniel McDuff, Cambridge, MA (US); and Mary P. Czerwinski, Kirkland, WA (US)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Oct. 30, 2020, as Appl. No. 17/085,428.
Prior Publication US 2022/0138583 A1, May 5, 2022
Int. Cl. G06N 3/088 (2023.01); G06N 3/04 (2023.01)
CPC G06N 3/088 (2013.01) [G06N 3/04 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A device comprising:
processing circuitry;
a memory including instructions that, when executed by the processing circuitry, cause the processing circuitry to perform operations, the operations comprising:
obtaining facial image data (FID) of an image of a first face;
adjusting the FID such that angle and pose of the face is consistent with corresponding angle and pose of faces in first facial image data samples of a variety of people resulting in adjusted FID;
manipulating, by a normalizing autoencoder, the adjusted FID resulting in normalized FID (NFID), the normalizing autoencoder (i) trained based on second facial image data samples of a template person different from the variety of people and the first facial image data samples and (ii) including (a) a single encoder trained based on both the first and second facial image data samples, (b) a first decoder trained based on only the second facial image data samples, and (c) a second decoder trained based on only the first facial image data samples, the manipulating including combining dynamic characteristics of the first face with static characteristics of a second face of the template person in the second facial image data samples, to generate the NFID, the static characteristics including characteristics that are the same among the second data samples;
providing the NFID as input to a facial action unit (FAU) classifier model to classify FAUs in the NFID based on the dynamic characteristics of the first face and the static characteristics of the second face; and
receiving, from the FAU classifier model, FAUs of the first face.