US 12,223,734 B2
Systems and methods for training machine-learned models with deviating intermediate representations
Xuanyuan Tu, Milton (CA); Raquel Urtasun, Toronto (CA); Tsun-Hsuan Wang, Cambridge (CA); Sivabalan Manivasagam, Toronto (CA); Jingkang Wang, Toronto (CA); and Mengye Ren, Toronto (CA)
Assigned to AURORA OPERATIONS, INC., Pittsburgh, PA (US)
Filed by Aurora Operations, Inc., Pittsburgh, PA (US)
Filed on Jan. 15, 2021, as Appl. No. 17/151,001.
Claims priority of provisional application 63/132,780, filed on Dec. 31, 2020.
Claims priority of provisional application 62/985,865, filed on Mar. 5, 2020.
Prior Publication US 2021/0279640 A1, Sep. 9, 2021
Int. Cl. G06V 20/56 (2022.01); G01S 17/931 (2020.01); G05D 1/00 (2024.01); G06F 18/21 (2023.01); G06F 18/24 (2023.01); G06N 20/00 (2019.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01)
CPC G06V 20/56 (2022.01) [G01S 17/931 (2020.01); G05D 1/0088 (2013.01); G05D 1/0221 (2013.01); G06F 18/2163 (2023.01); G06F 18/217 (2023.01); G06F 18/24 (2023.01); G06N 20/00 (2019.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method, the method comprising:
obtaining sensor data representative of a secondary environment proximate to an autonomous vehicle;
generating an intermediate representation for the autonomous vehicle based, at least in part, on the sensor data, wherein the intermediate representation is descriptive of at least a portion of the secondary environment;
determining an intermediate representation deviation for the intermediate representation based, at least in part, on the intermediate representation and a machine-learned model associated with the autonomous vehicle, by:
obtaining, via the machine-learned model, one or more ground truth bounding box proposals based, at least in part, on the intermediate representation,
obtaining, via the machine-learned model, one or more deviating bounding box proposals, and
determining the intermediate representation deviation for the intermediate representation based, at least in part, on a comparison between the one or more ground truth bound box proposals and the one or more deviating bounding box proposals;
generating data indicative of a modified intermediate representation based, at least in part, on the intermediate representation and the intermediate representation deviation; and
communicating the data indicative of the modified intermediate representation to a vehicle computing system associated with the autonomous vehicle.