US 12,141,995 B2
Systems and methods for simulating dynamic objects based on real world data
Ming Liang, Toronto (CA); Wei-Chiu Ma, Toronto (CA); Sivabalan Manivasagam, Toronto (CA); Raquel Urtasun, Toronto (CA); Bin Yang, Toronto (CA); and Ze Yang, Toronto (CA)
Assigned to AURORA OPERATIONS, INC., Pittsburgh, PA (US)
Filed by Aurora Operations, Inc., Pittsburgh, PA (US)
Filed on Jul. 29, 2021, as Appl. No. 17/388,372.
Claims priority of provisional application 63/058,052, filed on Jul. 29, 2020.
Prior Publication US 2022/0036579 A1, Feb. 3, 2022
Int. Cl. G06K 9/00 (2022.01); G06N 20/00 (2019.01); G06T 7/70 (2017.01); G06T 17/20 (2006.01); G06V 40/20 (2022.01)
CPC G06T 7/70 (2017.01) [G06N 20/00 (2019.01); G06T 17/20 (2013.01); G06V 40/23 (2022.01)] 18 Claims
OG exemplary drawing
 
1. A computing system comprising:
one or more processors; and
one or more computer-readable medium storing instructions that when executed by the one or more processors cause the computing system to perform operations, the operations comprising:
obtaining sensor data indicative of a dynamic object, wherein;
the sensor data comprises three-dimensional data corresponding to a first time and two-dimensional data corresponding to the first time, and
the three-dimensional data and two-dimensional data are associated with the dynamic object;
providing input data descriptive of the sensor data indicative of the dynamic object as an input to a machine-learned object parameter estimation model;
receiving as an output of the machine-learned object parameter estimation model, in response to receipt of the input data, a plurality of object model parameters indicative of at least one of a pose or a shape modification with respect to a template object model, wherein:
the plurality of object model parameters correspond to the first time, and
the plurality of object model parameters represent a three-dimensional dynamic object model indicative of an object pose and an object shape for the dynamic object;
obtaining sequential sensor data indicative of the dynamic object at one or more second times;
determining a sequence of object model parameters descriptive of a respective object pose for the dynamic object at the one or more second times, wherein the sequence of object model parameters is indicative of a trajectory of the dynamic object, wherein the trajectory is indicative of at least one or more velocities of the dynamic object; and
generating at least a portion of simulation data based, at least in part, on the sequence of object model parameters.