US 12,469,273 B2
Text-to-image diffusion model rearchitecture
Pavlo Chemerys, Amsterdam (NL); Colin Eles, Marina del Rey, CA (US); Ju Hu, Los Angeles, CA (US); Qing Jin, Palo Alto, CA (US); Yanyu Li, Malden, MA (US); Ergeta Muca, Long Island City, NY (US); Jian Ren, Marina Del Ray, CA (US); Dhritiman Sagar, Marina del Rey, CA (US); Aleksei Stoliar, Marina del Rey, CA (US); Sergey Tulyakov, Santa Monica, CA (US); and Huan Wang, Somerville, MA (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Dec. 29, 2023, as Appl. No. 18/400,873.
Claims priority of provisional application 63/504,563, filed on May 26, 2023.
Prior Publication US 2024/0394932 A1, Nov. 28, 2024
Int. Cl. G06N 20/00 (2019.01); G06N 3/0455 (2023.01); G06T 5/60 (2024.01); G06T 5/70 (2024.01); G06T 11/00 (2006.01); G06V 10/82 (2022.01); G10L 15/18 (2013.01); G10L 15/22 (2006.01)
CPC G06V 10/82 (2022.01) [G06N 3/0455 (2023.01); G06N 20/00 (2019.01); G06T 5/60 (2024.01); G06T 5/70 (2024.01); G06T 11/00 (2013.01); G10L 15/1815 (2013.01); G10L 15/22 (2013.01); G06T 2200/24 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
at least one processor; and
at least one memory component storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
identifying a performance characteristic for individual machine learning model blocks in an iterative denoising process of a machine learning model, wherein the performance characteristic includes a latency characteristic indicative of a time delay for the corresponding machine learning model block to process an input to generate an output, wherein the operations further comprise identifying the current machine learning model block based on the latency characteristic, wherein the current machine learning model block is subsequent to the prior machine learning model block and the current machine learning model block is prior to the subsequent machine learning model block;
identifying a current machine learning model block to remove from the machine learning model based on the identified performance characteristic;
identifying a prior machine learning model block connected to an input of the current machine learning model block and a subsequent machine learning model block connected to an output of the current machine learning model block;
identifying a prompt of a user, the prompt indicative of an intent of the user for generative images; and
analyzing data corresponding to the prompt using the machine learning model to generate one or more images, the machine learning model trained to generate images based on data corresponding to prompts.