US 11,704,912 B2
Label-free performance evaluator for traffic light classifier system
Guy Hotson, Mountain View, CA (US); Richard L. Kwant, San Bruno, CA (US); Brett Browning, Pittsburgh, PA (US); and Deva Ramanan, Pittsburgh, PA (US)
Assigned to Ford Global Technologies, LLC, Dearborn, MI (US)
Filed by FORD GLOBAL TECHNOLOGIES, LLC, Dearborn, MI (US)
Filed on Jun. 16, 2020, as Appl. No. 16/902,678.
Prior Publication US 2021/0390349 A1, Dec. 16, 2021
Int. Cl. B60W 40/04 (2006.01); B60W 60/00 (2020.01); G06F 18/22 (2023.01); G06F 18/214 (2023.01); G06F 18/2431 (2023.01); H04N 23/90 (2023.01); G06V 20/58 (2022.01); G06V 10/764 (2022.01); G06V 10/774 (2022.01); G06V 10/776 (2022.01); G06V 10/82 (2022.01); G06F 18/21 (2023.01)
CPC G06V 20/584 (2022.01) [B60W 40/04 (2013.01); B60W 60/0015 (2020.02); B60W 60/0025 (2020.02); B60W 60/0027 (2020.02); G06F 18/214 (2023.01); G06F 18/217 (2023.01); G06F 18/22 (2023.01); G06F 18/2431 (2023.01); G06V 10/764 (2022.01); G06V 10/776 (2022.01); G06V 10/7753 (2022.01); G06V 10/82 (2022.01); H04N 23/90 (2023.01); B60W 2420/42 (2013.01); B60W 2555/60 (2020.02)] 23 Claims
OG exemplary drawing
 
1. A method of evaluating a classifier used to determine a traffic light signal state in digital images, the method comprising:
by a computer vision system of a vehicle, receiving at least one digital image of a traffic signal device of an intersection, wherein the traffic signal device comprises a traffic signal face including one or more traffic signal elements; and
by a processor:
implementing a traffic light classifier (TLC) for the vehicle to determine a classification state of the traffic signal face based on the received at least one digital image,
evaluating a performance of the determining of the classification state by the TLC by applying one or more performance evaluator metrics to the classification state and without using labeled data from the at least one digital image,
identifying, in the evaluating, that the classification state determined by the TLC is a classification error,
generating label data for the at least one digital image based on the identified classification error;
relabeling the at least one digital image based on the generated label data, and
using the relabeled digital image to refine training of the TLC.