US 12,223,706 B2
Training energy-based models from a single image for internal learning and inference using trained models
Zilong Zheng, Beijing (CN); Jianwen Xie, Santa Clara, CA (US); and Ping Li, Bellevue, WA (US)
Assigned to Baidu USA LLC, Sunnyvale, CA (US)
Filed by Baidu USA, LLC, Sunnyvale, CA (US)
Filed on May 25, 2022, as Appl. No. 17/824,694.
Claims priority of provisional application 63/208,842, filed on Jun. 9, 2021.
Prior Publication US 2022/0398836 A1, Dec. 15, 2022
Int. Cl. G06V 10/82 (2022.01); G06V 10/774 (2022.01); G06V 10/84 (2022.01)
CPC G06V 10/85 (2022.01) [G06V 10/774 (2022.01); G06V 10/82 (2022.01)] 18 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
given a set of energy-based neural network models (EBMs) in which each EBM is configured to synthesize an image at a scale and the scales of the set of EBMs range from a minimal scale to a maximum scale, and given a multi-scale set of versions of a training image that similarly range from the minimal scale to the maximum scale so that each EBM has a corresponding version of the training image at the same scale as the EBM, performing steps comprising:
training each of the EBMs in the set of EBMs to synthesize an image at the scale of the EBM based on a patch distribution learned from the version of the training image at the scale of the EBM given an initial input image at the scale of the EBM and the version of the training image at the scale of the EBM, in which for at least some of the EBMs the initial input image is a synthesized image from a lower scaled EBM that has been upsampled to the scale of the EBM, comprising:
for each EBM:
using the initial input image for the EBM and the EBM in a Markov chain Monte Carlo (MCMC) sampling process to produce the synthesized image at the scale of the EBM; and
updating parameters of the EBM by using a comparison comprising one or more the synthesized image generated using the MCMC sampling process and the version of the training image at the scale of the EBM; and
responsive to a training stop condition being met, output the trained set of EBMs.