US 12,299,939 B2
Generating novel images using sketch image representations
Midhun Harikumar, Sunnyvale, CA (US); Pranav Aggarwal, Santa Clara, CA (US); and Ajinkya Gorakhnath Kale, San Jose, CA (US)
Assigned to Adobe Inc., San Jose, CA (US)
Filed by Adobe Inc., San Jose, CA (US)
Filed on Jun. 22, 2022, as Appl. No. 17/808,261.
Prior Publication US 2023/0419551 A1, Dec. 28, 2023
Int. Cl. G06T 7/10 (2017.01); G06T 9/00 (2006.01)
CPC G06T 9/001 (2013.01) [G06T 7/10 (2017.01); G06T 9/008 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method of generating a novel image, the method comprising:
generating, via a first machine learning model, a first sequence of coded representations of a first image having one or more features;
generating, via a second machine learning model, a second sequence of coded representations of a sketch image having one or more edge features associated with the one or more features of the first image, the sketch image comprising one or more sketch features provided directly over the one or more features of the first image;
predicting, via a third machine learning model, one or more subsequent coded representations iteratively based on the first sequence of coded representations of the first image, the second sequence of coded representations of the sketch image, and one or more previously iteratively predicted coded representations; and
based on the iterative predictions of the one or more subsequent coded representations, generating, via the third machine learning model, at least (i) a first portion of a reconstructed image which has one or more image attributes of the first image, and (ii) a second portion of the reconstructed image which is associated with the one or more edge features, the generating of the second portion of the reconstructed image comprising obtaining one or more new features based on the one or more sketch features.