US 11,727,724 B1
Emotion detection
Olivier Soares, San Jose, CA (US)
Assigned to Apple Inc., Cupertino, CA (US)
Filed by Apple Inc., Cupertino, CA (US)
Filed on Sep. 24, 2019, as Appl. No. 16/581,115.
Claims priority of provisional application 62/737,615, filed on Sep. 27, 2018.
Int. Cl. G06V 40/16 (2022.01); G06K 9/62 (2022.01); G06N 3/04 (2006.01); G06N 3/08 (2006.01); G06T 17/20 (2006.01)
CPC G06V 40/175 (2022.01) [G06K 9/6256 (2013.01); G06N 3/04 (2013.01); G06N 3/08 (2013.01); G06T 17/20 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A non-transitory computer readable medium comprising computer readable instructions executable by one or more processors to:
obtain a two dimensional (“2D”) image of at least part of a face;
apply, to the 2D image, an expression convolutional neural network (“CNN”) to obtain a latent vector for the face in the image, wherein the expression CNN predicts a latent vector representation of a three dimensional expression mesh based on an input 2D image;
compare values of the latent vector for the image to values of a plurality of previously processed latent vectors, wherein each of the plurality of previously processed latent vectors comprises a latent representation of a three dimensional (3D) expression mesh based on a previously processed 2D image, and wherein each of the plurality of previously processed latent vectors is associated with at least one emotion type; and
estimate an emotion type for the face based on a nearest match of the latent vectors among the previously processed latent vectors, wherein the estimated emotion type is selected from the emotion type associated with at least one of the plurality of previously processed latent vectors based on the match.