US 12,326,919 B2
Multiple stage image based object detection and recognition
Joseph Lawrence Amato, Pittsburgh, PA (US); Nemanja Djuric, Pittsburgh, PA (US); Shivam Gautam, Pittsburgh, PA (US); Abhishek Mohta, San Mateo, CA (US); and Fang-Chieh Chou, Redwood City, CA (US)
Assigned to AURORA OPERATIONS, INC., Pittsburgh, PA (US)
Filed by Aurora Operations, Inc., Pittsburgh, PA (US)
Filed on Apr. 29, 2022, as Appl. No. 17/733,688.
Application 17/733,688 is a continuation in part of application No. 17/007,969, filed on Aug. 31, 2020, granted, now 11,443,148.
Application 17/007,969 is a continuation of application No. 15/972,566, filed on May 7, 2018, granted, now 10,762,396, issued on Sep. 1, 2020.
Claims priority of provisional application 62/594,631, filed on Dec. 5, 2017.
Prior Publication US 2022/0261601 A1, Aug. 18, 2022
Int. Cl. G06K 9/00 (2022.01); G06F 18/214 (2023.01); G06F 18/241 (2023.01); G06F 18/243 (2023.01); G06N 7/01 (2023.01); G06N 20/00 (2019.01); G06T 7/521 (2017.01); G06T 15/08 (2011.01); G06V 10/28 (2022.01); G06V 10/50 (2022.01); G06V 10/56 (2022.01); G06V 20/58 (2022.01); G06V 20/64 (2022.01); G05D 1/00 (2006.01)
CPC G06F 18/241 (2023.01) [G06F 18/214 (2023.01); G06F 18/24323 (2023.01); G06N 7/01 (2023.01); G06N 20/00 (2019.01); G06T 7/521 (2017.01); G06T 15/08 (2013.01); G06V 10/28 (2022.01); G06V 10/50 (2022.01); G06V 10/56 (2022.01); G06V 20/58 (2022.01); G06V 20/584 (2022.01); G06V 20/64 (2022.01); G05D 1/0238 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/30261 (2013.01); G06T 2210/12 (2013.01)] 20 Claims
OG exemplary drawing
 
1. An autonomous vehicle control system for an autonomous vehicle, the autonomous vehicle control system comprising:
one or more processors; and
one or more non-transitory, computer-readable media storing instructions that are executable to cause the one or more processors to perform operations comprising:
receiving sensor data descriptive of an environment of the autonomous vehicle, the sensor data comprising a plurality of portions;
generating, by a first network of a machine-learned object detection and recognition model, a first classification value corresponding to a probability that a respective portion of the plurality of portions of sensor data corresponds to a foreground portion of the sensor data or a background portion of the sensor data;
generating, by a second network of the machine-learned object detection and recognition model, and based at least in part on the first classification value, a second classification value corresponding to a probability that the respective portion corresponds to one of one or more foreground classes or one or more background classes, the second network trained jointly with the first network; and
generating, based at least in part on the second classification value, an object output indicating detection of one or more objects in the sensor data.