US 12,307,628 B2
System and method for image transformation
Shuhui Jiang, Cambridge, MA (US); Aneesh Bhat, Cambridge, MA (US); Kai Ho Edgar Cheung, Malden, MA (US); and Yun Fu, Brookline, MA (US)
Assigned to Shiseido Company, Limited, Tokyo (JP)
Filed by Shiseido Company, Limited, Tokyo (JP)
Filed on Dec. 15, 2022, as Appl. No. 18/082,328.
Application 18/082,328 is a continuation of application No. PCT/IB2021/055768, filed on Jun. 28, 2021.
Claims priority of provisional application 63/047,335, filed on Jul. 2, 2020.
Prior Publication US 2023/0110393 A1, Apr. 13, 2023
Int. Cl. G06T 3/4046 (2024.01); G06T 5/40 (2006.01); G06T 5/70 (2024.01); G06T 11/00 (2006.01)
CPC G06T 3/4046 (2013.01) [G06T 5/40 (2013.01); G06T 5/70 (2024.01); G06T 11/001 (2013.01); G06T 2200/24 (2013.01); G06T 2207/10016 (2013.01); G06T 2207/10024 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30201 (2013.01)] 23 Claims
OG exemplary drawing
 
1. A method for training a neural network for image transformation, the method comprising:
receiving an input image including a predefined region, the predefined region having a baseline characteristic;
generating, using a generator, a modified image based on the input image, the modified image having a selected characteristic that is different than the baseline characteristic;
reducing a resolution of the modified image to produce a reduced resolution image;
generating, using a baseline generator and based on the reduced resolution image, a generated image having the baseline characteristic;
constructing a loss function based upon a comparison of the generated image and the input image;
optimizing the loss function by applying the loss function to the baseline generator to generate at least one subsequent image if the loss function exceeds a desired optimization;
receiving a second input image including a second predefined region, the second predefined region having the selected characteristic;
generating, using the baseline generator, a second generated image based on the second input image;
generating, using the generator, a second modified image based on the second generated image;
constructing a second loss function based upon a comparison of the second modified image and the second input image:
optimizing the second loss function by applying the second loss function to the baseline generator to generate at least one sub sequent image if the loss function exceeds a desired optimization;
adding noise to the second input image to generate a noise image;
generating, using the baseline generator, a generated noise image based on the noise image, the generated noise image and the second generated image having the baseline characteristic;
constructing a noise function based on a comparison of the generated noise image and the second generated image; and
updating the second loss function associated with the baseline generator with the noise function.