US 11,656,620 B2
Generating environmental parameters based on sensor data using machine learning
Dmytro Trofymov, Los Altos, CA (US); Pranav Maheshwari, Palo Alto, CA (US); and Vahid R. Ramezani, Portola Valley, CA (US)
Assigned to Luminar, LLC, Orlando, FL (US)
Filed by Luminar, LLC, Orlando, FL (US)
Filed on Mar. 6, 2019, as Appl. No. 16/294,274.
Claims priority of provisional application 62/787,163, filed on Dec. 31, 2018.
Prior Publication US 2020/0209858 A1, Jul. 2, 2020
Int. Cl. G05D 1/00 (2006.01); G06N 20/00 (2019.01); G06N 3/08 (2023.01); G05D 1/02 (2020.01); B60W 60/00 (2020.01)
CPC G05D 1/0088 (2013.01) [B60W 60/00 (2020.02); G05D 1/0242 (2013.01); G05D 1/0246 (2013.01); G05D 1/0257 (2013.01); G06N 3/08 (2013.01); G06N 20/00 (2019.01); B60K 2370/175 (2019.05); G05D 2201/0213 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for generating a machine learning model for controlling autonomous vehicles, the method comprising:
obtaining, by processing hardware, first training sensor data from a plurality of real-world sensors associated with one or more training vehicles, the first training sensor data being indicative of physical conditions of an environment in which the one or more training vehicles operate;
obtaining, by the processing hardware, second training sensor data from a plurality of virtual sensors, the second training sensor data being indicative of simulated physical conditions of a virtual environment; and
training, by the processing hardware, a machine learning (ML) model using both real-world and virtual training datasets including the first training sensor data, the second training sensor data, and respective sensor setting parameters of the plurality of real-world sensors associated with the one or more training vehicles and the plurality of virtual sensors, the ML model being trained for generating physical environment parameters in response to input of real-world sensor output data and associated real-world sensor setting parameters, wherein the real-world and virtual training datasets used to train the ML model include indications of which subsets of sensor data correspond to which of the respective sensor parameter settings including one or more of the following: different scan line distributions or different exposure settings;
wherein a controller in an autonomous vehicle (i) receives new sensor data from one or more sensors operating in the autonomous vehicle, (ii) receives respective current setting parameters from the one or more sensors of the autonomous vehicle, (iii) applies the received new sensor data and the received respective current sensor setting parameters to the ML model to generate current parameters of a current environment in which the autonomous vehicle operates, (iv) provides the generated current parameters to a motion planner component to generate decisions for controlling the autonomous vehicle, and (v) causes the autonomous vehicle to maneuver in accordance with the generated decisions.