US 11,734,889 B2
Method of gaze estimation with 3D face reconstructing
Feng Lu, Beijing (CN); and Jiadong Liang, Beijing (CN)
Assigned to Beihang University, Beijing (CN)
Filed by Beihang University, Beijing (CN)
Filed on Apr. 13, 2022, as Appl. No. 17/719,948.
Claims priority of application No. 202111376077.5 (CN), filed on Nov. 19, 2021.
Prior Publication US 2022/0254106 A1, Aug. 11, 2022
Int. Cl. G06T 17/20 (2006.01); G06V 40/16 (2022.01); G06V 10/44 (2022.01); G06V 10/82 (2022.01); G06V 40/18 (2022.01); G06T 15/04 (2011.01); G06T 19/20 (2011.01)
CPC G06T 17/20 (2013.01) [G06T 15/04 (2013.01); G06T 19/20 (2013.01); G06V 10/454 (2022.01); G06V 10/82 (2022.01); G06V 40/168 (2022.01); G06V 40/193 (2022.01); G06T 2219/2016 (2013.01)] 7 Claims
OG exemplary drawing
 
1. A method of gaze estimation with 3D face reconstructing, comprising:
step 1, extracting reconstruction coefficients from each reference 2D face image, and reconstructing a corresponding basic 3D face according to a 3D morphable model;
step 2, for each basic 3D reconstructed face, optimizing local area details to obtain a shape-aligned 3D reconstructed face;
step 3, for each shape-aligned 3D reconstructed face, aligning its eye area texture with the reference 2D face image to obtain a gaze-aligned 3D reconstructed face;
step 4, using photos of the 3D reconstructed face at various rotation angles and designing a specific gradient loss function to train a gaze contact discriminator;
step 5, for each gaze-aligned 3D reconstructed face waiting to achieve gaze contact, using the gaze contact discriminator to calculate a gradient of its gaze direction after rotation in each direction, offset from a gaze contact direction, designing a specific rotation method, and rotating the 3D reconstructed face continuously until the gaze contact is completed.