US 12,323,571 B2
Method of training a neural network configured for converting 2D images into 3D models
Vsevolod Kagarlitsky, Ramat Gan (IL); Shirley Keinan, Tel Aviv (IL); Michael Birnboim, Holon (IL); Amir Green, Mitzpe Netofa (IL); Alik Mokeichev, Tel Aviv (IL); Michal Heker, Tel-Aviv (IL); Yair Baruch, Tel Aviv (IL); Gil Wohlstadter, Givataim (IL); Gilad Talmon, Givataim (IL); and Michael Tamir, Tel Aviv (IL)
Assigned to TETAVI LTD., Tel-Aviv (IL)
Filed by TETAVI LTD., Tel-Aviv (IL)
Filed on Dec. 30, 2021, as Appl. No. 17/566,623.
Claims priority of provisional application 63/134,207, filed on Jan. 6, 2021.
Prior Publication US 2022/0217321 A1, Jul. 7, 2022
Int. Cl. H04N 13/275 (2018.01); G06T 7/579 (2017.01); H04N 13/172 (2018.01)
CPC H04N 13/275 (2018.05) [G06T 7/579 (2017.01); H04N 13/172 (2018.05); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 15 Claims
OG exemplary drawing
 
1. A computer-implemented method of generating a database for training a neural network, said neural network configured, when trained, for converting 2d images into 3d models; said database comprising a training collection of rendered 2d images and 3d models corresponding thereto; said method comprising steps of:
a. obtaining said 3d models by means of at least one of the following:
i. capturing a number of single volumetric frames of said at least one character in static poses and obtaining said 3d model from said number of single volumetric frames; and
ii. capturing a volumetric video of said at least one character being in motion and obtaining said 3d model from said volumetric video;
said at least one character at sub-steps i to iii being identical to each other or differ from each other;
b. generating said rendered 2d image by rendering said 3d model in a 2d image format from at least one view point; and
c. generating said database by collecting pairs, each of said pairs comprising said rendered 2d image and a corresponding sampled 3d model, said corresponding sampled 3d model comprising at least a portion of said 3d model generated at step (a);
d. said database training at least one neural network configured to convert at least one 2d image to a 3d model;
wherein said 3d model obtained at step (a) is an undivided 3d model, obtained directly, without a dummy model.