US 12,469,180 B2
Machine learning-based modification of image content
Celia Nicole Mourkogiannis, Los Angeles, CA (US)
Assigned to Snap Inc, Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Mar. 16, 2021, as Appl. No. 17/202,851.
Claims priority of provisional application 63/000,220, filed on Mar. 26, 2020.
Prior Publication US 2021/0304449 A1, Sep. 30, 2021
Int. Cl. G06T 11/00 (2006.01); G06F 3/0482 (2013.01); G06N 20/00 (2019.01); H04L 51/10 (2022.01); H04L 51/52 (2022.01)
CPC G06T 11/00 (2013.01) [G06F 3/0482 (2013.01); G06N 20/00 (2019.01); H04L 51/10 (2013.01); H04L 51/52 (2022.05); G06T 2200/24 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method, comprising:
providing, by a messaging application running on a device, a message preview interface for displaying an image captured by a camera of the device,
wherein the message preview interface includes a set of editing tools, each tool within the set of editing tools being represented by a respective icon which is user selectable to annotate or modify the image, and
wherein the message preview interface further includes a plurality of content modifiers is separate from the set of editing tools, each content modifier within the plurality of content modifiers corresponding to respective augmented reality content that is automatically applied to the image via respective swipe gestures to individually cycle through different augmented reality content to apply to the image, the respective swipe gestures being performed directly on the image, and
wherein the plurality of content modifiers includes a first content modifier configured to apply augmented reality content to the image using a machine learning model, the machine learning model having been trained with a plurality of image pairs, each image pair including a first image and a second image corresponding to a modified version of the first image;
receiving, by the messaging application, a swipe gesture selecting the first content modifier from among the plurality of content modifiers;
determining, in response to receiving the swipe gesture, a modified version of the image based on output from the machine learning model; and
displaying the modified version of the image.