US 12,124,803 B2
Text-guided sticker generation
Arnab Ghosh, Oxford (GB); Jian Ren, Marina Del Ray, CA (US); Pavel Savchenkov, London (GB); and Sergey Tulyakov, Marina del Rey, CA (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Aug. 17, 2022, as Appl. No. 17/820,437.
Prior Publication US 2024/0062008 A1, Feb. 22, 2024
Int. Cl. G06F 40/289 (2020.01); G06F 3/04842 (2022.01); G06F 16/583 (2019.01); G06T 11/60 (2006.01); H04L 51/10 (2022.01)
CPC G06F 40/289 (2020.01) [G06F 3/04842 (2013.01); G06F 16/5846 (2019.01); G06T 11/60 (2013.01); H04L 51/10 (2013.01); G06T 2200/24 (2013.01)] 17 Claims
OG exemplary drawing
 
1. A computer-implemented method of generating an image for use in a conversation taking place in a messaging application, the method comprising:
receiving conversation input text from a user of a portable device that includes a display;
expanding the conversation input text by generating an image caption from the conversation input text with a natural language processing (NLP)-based creative caption model that has been trained to generate image captions from short queries;
generating model input text from the expanded conversation input text;
generating, by a text-to-image model, an image from at least some of the model input text, the text-to-image model having been generated from a large scale image dataset of general images and text pairs and refined by an existing collection of sticker graphical elements for application like a label to a media item or for inclusion with a messaging conversation;
displaying the image on the display of the portable device;
receiving user input to transmit the image in a message; and
transmitting, in response to receiving the user input, the image to a remote recipient.