US 11,854,212 B2
Traffic light detection system for vehicle
Maria Antoinette Meijburg, Boston, MA (US); Paul Schmitt, Merrimack, NH (US); and Philipp Robbel, Cambridge, MA (US)
Assigned to Motional AD LLC, Boston, MA (US)
Filed by Motional AD LLC, Boston, MA (US)
Filed on Nov. 3, 2020, as Appl. No. 17/088,001.
Claims priority of provisional application 62/981,597, filed on Feb. 26, 2020.
Prior Publication US 2021/0261152 A1, Aug. 26, 2021
Int. Cl. G06T 7/00 (2017.01); G06T 7/246 (2017.01); B60W 60/00 (2020.01); H04W 4/44 (2018.01); G08G 1/01 (2006.01); G06T 7/292 (2017.01); G01S 19/42 (2010.01); G01C 21/34 (2006.01); G01C 21/36 (2006.01); H04W 4/80 (2018.01); G06V 20/58 (2022.01); G06F 18/214 (2023.01); B60W 30/18 (2012.01)
CPC G06T 7/246 (2017.01) [B60W 60/001 (2020.02); G01C 21/3492 (2013.01); G01C 21/3691 (2013.01); G01S 19/42 (2013.01); G06F 18/214 (2023.01); G06T 7/292 (2017.01); G06V 20/584 (2022.01); G08G 1/0116 (2013.01); H04W 4/44 (2018.02); H04W 4/80 (2018.02); B60W 30/18154 (2013.01); B60W 2420/42 (2013.01); B60W 2420/52 (2013.01); B60W 2555/60 (2020.02); G06T 2207/10016 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30252 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A method comprising:
detecting, by one or more processors of a vehicle, a traffic light located at a first spatiotemporal location based on a first digital video stream captured by a first camera of the vehicle and a second digital video stream captured by a second camera of the vehicle;
determining, by the one or more processors, that the vehicle is located at a second spatiotemporal location, the determining comprising validating, by the one or more processors, first location data obtained from a plurality of sensors of the vehicle against second location data obtained by filtering, by a filter, the first location data;
determining, by the one or more processors, that the traffic light is expected at the first spatiotemporal location based on a semantic map referenced by the second spatiotemporal location;
responsive to determining that the traffic light is expected at the first spatiotemporal location, detecting, by the one or more processors, a traffic signal of the traffic light based on the first digital video stream and the second digital video stream, wherein detecting the traffic signal comprises:
extracting, by the one or more processors, a feature vector from the first digital video stream and the second digital video stream; and
determining, based on a machine learning model executed by the one or more processors, the traffic signal based on the feature vector, the machine learning model trained to recognize a color of an object based on features extracted from digital video streams of the object, wherein the machine learning model comprises a first artificial neural network trained to recognize red lights and a second artificial neural network trained to recognize green lights, the first artificial neural network independent of the second artificial neural network;
determining, by the one or more processors, a trajectory of the vehicle in accordance with the traffic signal; and
causing the vehicle to operate in accordance with the determined trajectory.