US 11,989,651 B2
Method and system for on-the-fly object labeling via cross modality validation in autonomous driving vehicles
Hao Zheng, Saratoga, CA (US); David Wanqian Liu, Los Altos, CA (US); and Timothy Patrick Daly, Jr., San Jose, CA (US)
Assigned to PlusAI, Inc., Santa Clara, CA (US)
Filed by PlusAI, Inc., Santa Clara, CA (US)
Filed on Dec. 7, 2022, as Appl. No. 18/077,022.
Application 18/077,022 is a continuation of application No. 15/856,215, filed on Dec. 28, 2017, granted, now 11,537,126.
Application 15/856,215 is a continuation of application No. 15/615,284, filed on Jun. 6, 2017, granted, now 11,042,155, issued on Jun. 22, 2021.
Prior Publication US 2023/0096020 A1, Mar. 30, 2023
Int. Cl. G06N 20/00 (2019.01); B60W 30/00 (2006.01); G05D 1/00 (2006.01); G06F 18/21 (2023.01); G06F 18/25 (2023.01); G06N 3/08 (2023.01); G06N 3/092 (2023.01); G06N 3/098 (2023.01); G06N 5/02 (2023.01); G06N 7/01 (2023.01); G06V 10/774 (2022.01); G06V 10/776 (2022.01); G06V 10/80 (2022.01); G06V 20/56 (2022.01); G06V 20/58 (2022.01)
CPC G06N 3/08 (2013.01) [B60W 30/00 (2013.01); G05D 1/0088 (2013.01); G05D 1/0274 (2013.01); G05D 1/0287 (2013.01); G06F 18/217 (2023.01); G06F 18/251 (2023.01); G06N 3/092 (2023.01); G06N 3/098 (2023.01); G06N 5/02 (2013.01); G06N 7/01 (2023.01); G06N 20/00 (2019.01); G06V 10/774 (2022.01); G06V 10/776 (2022.01); G06V 10/803 (2022.01); G06V 20/56 (2022.01); G06V 20/58 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
receiving, by a computing system, a data package that includes a set of sensor data captured at an environment;
determining, by the computing system, a first subset of the sensor data in the data package associated with first confidence scores that satisfy a confidence threshold for detected features and a second subset of the sensor data associated with second confidence scores that fail to satisfy the confidence threshold;
updating, by the computing system, a global model based on the first subset of the sensor data in the data package;
generating, by the computing system, an update for local class models based on the second subset of the sensor data, wherein the second subset of the sensor data is modified to include labels for occluded regions in the second subset of the sensor data that correspond to features detected in the first subset of the sensor data based on a temporal relationship between the second subset of the sensor data and the first subset of the sensor data;
generating, by the computing system, adaptations of the updated global model based on class model configurations associated with the local class models, wherein at least one of the adaptations includes the update for the local class models, wherein at least one of the class model configurations is associated with a weather related class model of the local class models;
providing, by the computing system, the adaptations to the local class models based on the class model configurations; and
causing, by the computing system, control of a vehicle in the environment based on at least one of the local class models.