US 11,727,169 B2
Systems and methods for inferring simulated data
Randall J. St. Romain, II, Ann Arbor, MI (US); Hiroyuki Funaya, Ann Arbor, MI (US); and Michael James Delp, Ann Arbor, MI (US)
Assigned to Toyota Research Institute, Inc., Los Altos, CA (US)
Filed by Toyota Research Institute, Inc., Los Altos, CA (US)
Filed on Sep. 11, 2019, as Appl. No. 16/567,297.
Prior Publication US 2021/0073345 A1, Mar. 11, 2021
Int. Cl. G06F 30/20 (2020.01); G06N 3/08 (2023.01); G06N 3/04 (2023.01); G05D 1/00 (2006.01); G05D 1/02 (2020.01)
CPC G06F 30/20 (2020.01) [G06N 3/04 (2013.01); G06N 3/08 (2013.01); G05D 1/0088 (2013.01); G05D 1/0231 (2013.01); G05D 2201/0213 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A simulation system for simulating aspects of sensor data, comprising:
one or more processors;
a memory communicably coupled to the one or more processors and storing:
an input module including instructions that when executed by the one or more processors cause the one or more processors to, in response to receiving a request to generate simulated information associated with the sensor data, acquire the sensor data from at least one sensor that includes at least range information about a perceived environment observed by the at least one sensor, and to generate semantic labels for objects in the perceived environment that are depicted by the sensor data, wherein the request for the simulated information specifies one or more attributes of the sensor data that are absent from the sensor data in a current format; and
a simulation module including instructions that when executed by the one or more processors cause the one or more processors to:
compute the simulated information associated with the sensor data using a machine learning model that accepts the sensor data and the semantic labels as an input and processes the input, including the semantic labels, to produce the simulated information as an output that completes the sensor data, and
provide the simulated information with the sensor data,
wherein the simulation module includes instructions to:
train the machine learning model using training data that includes training range data and training intensity data by comparing the training intensity data with the simulated information that includes inferred intensity data produced by the machine learning model to determine a loss.