US 11,869,149 B2
Computer-based techniques for learning compositional representations of 3D point clouds
Ben Eckart, Oakland, CA (US); Christopher Choy, Los Angeles, CA (US); Chao Liu, Pittsburgh, PA (US); and Yurong You, Ithaca, NY (US)
Assigned to NVIDIA Corporation, Santa Clara, CA (US)
Filed by NVIDIA CORPORATION, Santa Clara, CA (US)
Filed on May 13, 2022, as Appl. No. 17/744,467.
Prior Publication US 2023/0368468 A1, Nov. 16, 2023
Int. Cl. G06T 17/10 (2006.01); G06T 19/20 (2011.01); G06N 3/084 (2023.01); G06N 3/045 (2023.01)
CPC G06T 17/10 (2013.01) [G06N 3/045 (2023.01); G06N 3/084 (2013.01); G06T 19/20 (2013.01); G06T 2219/2016 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method for training a machine learning model to generate representations of point clouds, the method comprising:
executing a first neural network on a first point cloud that represents a first three-dimensional (3D) scene to generate a key set and a value set;
generating an output vector set based on a first query set, the key set, and the value set;
computing a plurality of spatial features based on the output vector set;
computing a plurality of quantized context features based on the output vector set and a first set of codes representing a first set of 3D geometry blocks; and
modifying the first neural network based on a likelihood of reconstructing the first point cloud, the plurality of quantized context features, and the plurality of spatial features to generate an updated neural network,
wherein a trained machine learning model includes the updated neural network, a second query set, and a second set of codes representing a second set of 3D geometry blocks and maps a point cloud representing a 3D scene to a representation of a plurality of 3D geometry instances.