US 12,020,383 B2
Facial synthesis in augmented reality content for third party applications
Grigoriy Tkachenko, London (GB); and Inna Zaitseva, Kiev (UA)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Mar. 30, 2022, as Appl. No. 17/708,744.
Claims priority of provisional application 63/200,882, filed on Mar. 31, 2021.
Prior Publication US 2022/0319127 A1, Oct. 6, 2022
Int. Cl. G06T 19/00 (2011.01); G06F 3/0482 (2013.01); G06T 7/70 (2017.01); G06V 40/16 (2022.01)
CPC G06T 19/006 (2013.01) [G06F 3/0482 (2013.01); G06T 7/70 (2017.01); G06V 40/174 (2022.01); G06T 2200/24 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30201 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method, comprising:
providing for display a first graphical interface in a third party application, the first graphical interface including at least a first selectable graphical item for text entry, the third party application being executed by a computing device separate from a first party application and a messaging server system associated with the first party application, the first party application and the messaging server system corresponding to a same entity;
receiving a selection of the first selectable graphical item;
in response to receiving the selection of the first selectable graphical item, providing for display a second graphical interface in the third party application, the second graphical display including at least a second selectable graphical item to initiate generating augment reality content including facial synthesis;
receiving, by one or more hardware processors, a selection of the second selectable graphical item to initiate generating augment reality content including facial synthesis, the selection of the second selectable graphical item being received by the third party application;
capturing, by one or more hardware processors, image data by the computing device, the captured image data comprising a target face of a target actor and facial expressions of the target actor;
generating, by the one or more hardware processors and based at least in part on frames of a source media content, sets of source pose parameters, the sets of the source pose parameters comprising positions of representations of a head of a source actor and facial expressions of the source actor in the frames of the source media content;
generating, based at least in part on sets of the source pose parameters, by the one or more hardware processors, an output media content using an interface communicating with the messaging server system, each frame of the output media content including an image of the target face, from the captured image data, in at least one frame of the output media content, the image of the target face being modified based on at least one of the sets of the source pose parameters to mimic at least one of positions of the head of the source actor and facial expressions in the frames of the source media content; and
providing, by the one or more hardware processors, augmented reality content based at least in part on the output media content for display on the computing device.