US 12,462,492 B2
Beautification techniques for 3D data in a messaging system
Kyle Goodrich, Venice, CA (US); Samuel Edward Hare, Los Angeles, CA (US); Maxim Maximov Lazarov, Culver City, CA (US); Tony Mathew, Los Angeles, CA (US); Andrew James McPhee, Culver City, CA (US); Daniel Moreno, New York, NY (US); Dhritiman Sagar, Marina del Rey, CA (US); and Wentao Shang, Los Angeles, CA (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Aug. 31, 2023, as Appl. No. 18/240,380.
Application 18/240,380 is a continuation of application No. 17/878,635, filed on Aug. 1, 2022, granted, now 11,776,233.
Application 17/878,635 is a continuation of application No. 17/006,471, filed on Aug. 28, 2020, granted, now 11,410,401.
Claims priority of provisional application 62/893,046, filed on Aug. 28, 2019.
Prior Publication US 2023/0410450 A1, Dec. 21, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 19/00 (2011.01); G06F 3/04842 (2022.01); G06F 3/04883 (2022.01); G06N 20/00 (2019.01); G06T 7/194 (2017.01); G06T 7/50 (2017.01); G06T 7/507 (2017.01); G06T 15/50 (2011.01); G06T 19/20 (2011.01); G06V 40/16 (2022.01); H04L 67/131 (2022.01)
CPC G06T 19/006 (2013.01) [G06F 3/04842 (2013.01); G06F 3/04883 (2013.01); G06N 20/00 (2019.01); G06T 7/194 (2017.01); G06T 7/50 (2017.01); G06T 7/507 (2017.01); G06T 15/50 (2013.01); G06T 19/20 (2013.01); G06V 40/171 (2022.01); H04L 67/131 (2022.05); G06T 2207/10028 (2013.01); G06T 2207/30201 (2013.01); G06T 2219/2012 (2013.01); G06T 2219/2024 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
applying, to image data and depth data, a 3D effect including at least one beautification operation based at least in part on an augmented reality content generator, the 3D effect including at least one beautification operation, the beautification operation comprising modifying image data, the image data including a region corresponding to a representation of a face, the beautification operation comprising using a machine learning model for at least one of smoothing blemishes or preserving facial skin texture, and the applying comprising:
generating a depth map using at least the depth data,
generating a segmentation mask based at least on the image data, and
performing background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data; and
generating a 3D message based at least in part on the applied 3D effect including the at least one beautification operation.