US 11,953,896 B2
System and method for a modular and continually learning remote guidance system for autonomous vehicles
John Russell Lepird, Pittsburgh, PA (US); and Artem Churkin, Munich (DE)
Assigned to Argo AI, LLC, Pittsburgh, PA (US)
Filed by Argo AI, LLC, Pittsburgh, PA (US)
Filed on Feb. 15, 2021, as Appl. No. 17/175,814.
Prior Publication US 2022/0260989 A1, Aug. 18, 2022
Int. Cl. G05D 1/00 (2006.01); B60W 50/02 (2012.01); B60W 50/06 (2006.01); B60W 60/00 (2020.01)
CPC G05D 1/0011 (2013.01) [B60W 50/0205 (2013.01); B60W 50/06 (2013.01); B60W 60/001 (2020.02); B60W 2420/42 (2013.01); B60W 2420/52 (2013.01); B60W 2556/45 (2020.02)] 16 Claims
OG exemplary drawing
 
1. A computer-implemented method for controlling at least one driving operation of an autonomous vehicle, comprising:
obtaining sensor data associated with an environment surrounding the autonomous vehicle;
providing the sensor data to a plurality of plugins, wherein each plugin of the plurality of plugins is configured to determine an existence of a different autonomy anomaly than the other plugins of the plurality of plugins;
independently determining, with each plugin of the plurality of plugins, based on the sensor data, whether to initiate a remote guidance session for the autonomous vehicle due to the existence of the autonomy anomaly associated with that plugin, wherein each plugin of the plurality of plugins includes a different model that is applied by that plugin to the sensor data to determine whether to request the remote guidance session;
receiving, from at least one plugin of the plurality of plugins, a request to initiate the remote guidance session for the autonomous vehicle;
in response to receiving the request to initiate the remote guidance session, automatically communicating with a computing device external to the autonomous vehicle to establish the remote guidance session with the computing device, wherein the remote guidance session provides, via the computing device, a human operator with information associated with the environment surrounding the autonomous vehicle and an ability to control at least one driving operation of the autonomous vehicle;
receiving, from the computing device external to the autonomous vehicle, during the remote guidance session, remote guidance data; and
controlling, based on the remote guidance data, during the remote guidance session, the at least one driving operation of the autonomous vehicle.