US 11,789,527 B1
Eyewear device external face tracking overlay generation
Kostiantyn Bilous, Santa Monica, CA (US); and Stanislav Minakov, Playa Vista, CA (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Kostiantyn Bilous, Santa Monica, CA (US); and Stanislav Minakov, Playa Vista, CA (US)
Filed on Nov. 17, 2020, as Appl. No. 16/950,045.
Claims priority of provisional application 62/948,986, filed on Dec. 17, 2019.
Int. Cl. G06F 3/01 (2006.01); G09G 5/377 (2006.01); G06N 20/00 (2019.01); G02B 27/01 (2006.01); G06V 40/16 (2022.01); G06F 18/214 (2023.01)
CPC G06F 3/012 (2013.01) [G02B 27/0172 (2013.01); G06F 18/214 (2023.01); G06N 20/00 (2019.01); G06V 40/161 (2022.01); G06V 40/171 (2022.01); G09G 5/377 (2013.01); G02B 2027/014 (2013.01); G02B 2027/0138 (2013.01); G02B 2027/0178 (2013.01); G09G 2354/00 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A system comprising:
an eyewear device, including:
a frame;
a temple extending from a lateral side of the frame;
an image display to present an overlay image to a user of the eyewear device;
a camera connected to the frame or the temple to capture an initial image of a scene the user of the eyewear device is viewing from a first perspective, the scene including a face;
another camera connected to the frame, the temple, or another temple and positioned to capture another initial image of the scene including the face from another perspective;
a processor coupled to the eyewear device and connected to the camera and the other camera;
a memory accessible to the processor; and
programming in the memory, wherein execution of the programming by the processor configures the system to perform functions, including functions to:
capture, via the camera, the initial image of the scene including the face;
capture, via the other camera, the other initial image of the scene including the face;
identify the face in the initial image of the scene;
identify the face in the other initial image of the scene;
identify facial feature parameters of the face in the initial image corresponding to an apparent size of the face in the initial image;
identify the facial feature parameters of the face in the other initial image;
track positional information of the face with respect to the eyewear device by comparing the identified facial feature parameters in both the initial image and the other initial image to a model trained using a plurality of images including faces having facial features with known positional information;
generate the overlay image responsive to the tracked positional information; and
present the overlay image on the image display.