US 11,809,484 B2
Training of differentiable renderer and neural network for query of 3D model database
Benjamin Planche, New Brunswick, NJ (US); and Rajat Vikram Singh, Santa Clara, CA (US)
Assigned to Siemens Industry Software Inc., Plano, TX (US)
Appl. No. 17/905,260
Filed by Siemens Industry Software Inc., Plano, TX (US)
PCT Filed Aug. 28, 2020, PCT No. PCT/US2020/048309
§ 371(c)(1), (2) Date Aug. 30, 2022,
PCT Pub. No. WO2021/178000, PCT Pub. Date Sep. 10, 2021.
Claims priority of provisional application 62/984,888, filed on Mar. 4, 2020.
Prior Publication US 2023/0111048 A1, Apr. 13, 2023
Int. Cl. G06F 16/56 (2019.01); G06F 16/55 (2019.01); G06N 20/00 (2019.01)
CPC G06F 16/56 (2019.01) [G06F 16/55 (2019.01); G06N 20/00 (2019.01)] 12 Claims
OG exemplary drawing
 
1. A system of differentiable networks trainable to learn an optimized query of a 3D model database used for object recognition, the system comprising:
a processor; and
a non-transitory memory having stored thereon modules executed by the processor, the modules comprising:
a database of a virtual representation data for a plurality 3D models of objects, the data configured as a list of points and vectors representing faces of the objects in 3D space;
an end-to-end differentiable pipeline comprising a first differentiable network and a second differentiable network,
wherein the first differentiable network is configured as a differentiable renderer to be trained to generate a first set of 2D images from a first 3D model of a first object and a second set of 2D images from a second 3D model of a second object, the first set of 2D images being rendered according to optimized rendering parameters for producing 2D images, the rendering parameters determined by gradient descent of a first triple loss function with optimization (i) to maximize visual variation among N images generated for the first 3D model and (ii)) to maximize visual dissimilarity between N images generated for the first model and N images generated for the second model; and
wherein the second differentiable network is configured as a convolutional neural network defined by a regression function and to be trained to generate searchable features of the first set of 2D images, the features configured as feature vectors extracted from the first set of 2D images according to optimized neural network parameters determined by gradient descent of a second triple loss function with optimization to achieve high correlation to an input image of the first set of 2D images and low correlation to images of the second set of 2D images.