US 12,392,686 B2
Instantiating objects in a simulated environment based on log data
Alireza Ashayer, South San Francisco, CA (US); Ryan Martin Cahoon, Mountain View, CA (US); and Rasoul Kabirzadeh, Redwood, CA (US)
Assigned to Zoox, Inc., Foster City, CA (US)
Filed by Zoox, Inc., Foster City, CA (US)
Filed on Mar. 5, 2021, as Appl. No. 17/193,826.
Prior Publication US 2022/0283055 A1, Sep. 8, 2022
Int. Cl. G01M 17/007 (2006.01); B60W 60/00 (2020.01)
CPC G01M 17/007 (2013.01) [B60W 60/0011 (2020.02); B60W 2556/10 (2020.02)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
receiving log data representing a real environment in which a real vehicle was operating during a period of time, wherein the log data comprises:
a plurality of real timestamps along a real timeline;
first data describing the real environment at the plurality of real timestamps; and
second data describing the real vehicle at the plurality of real timestamps;
generating, using the log data, a simulated scenario for testing an autonomous vehicle controller, the simulated scenario including a simulated environment representing at least a portion of the real environment and a simulated timeline in the simulated scenario;
causing, by the autonomous vehicle controller, a simulated autonomous vehicle to traverse the simulated environment, wherein a simulated path of the simulated autonomous vehicle in the simulated environment deviates, as a deviated simulated path, from a real path traversed by the real vehicle in the real environment;
generating, based at least on the simulated timeline, a state of the simulated environment, wherein the state of the simulated environment comprises a representation of a portion of the simulated environment at a time associated with the state and the portion of the simulated environment is based at least on the simulated autonomous vehicle at the time associated with the state;
determining, based at least on the deviated simulated path, a simulated location of the simulated autonomous vehicle in the simulated environment at a simulation timestamp, wherein the simulation timestamp indicates a time along the simulated timeline corresponding to the time associated with the state of the simulated environment;
determining, based at least in part on the simulated location and the log data, a real location of the real vehicle in the real environment which corresponds closest with the simulated location of the simulated autonomous vehicle in the simulated environment, wherein the real location of the real vehicle is different from the simulated location based at least on the deviated simulated path deviating from the real path;
determining, based at least on the log data and the real location, a respective timestamp of the plurality of real timestamps indicating a time along the real timeline associated with the real location of the real vehicle, wherein the time indicated by the simulation timestamp along the simulated timeline is different than the time indicated by the respective timestamp along the real timeline;
obtaining, from the log data, object data representing a real object in the real environment that was perceived by the real vehicle at the respective timestamp; and
updating, in the simulated environment, at the simulation timestamp, and based at least on the object data, a simulated object representing the real object at the respective timestamp.