US 12,001,519 B2
Object classification and related applications based on frame and event camera processing
Ted Yamazaki, San Diego, CA (US); Tonni Larsen, San Diego, CA (US); Leigh Anderson, Novi, MI (US); Tatsushi Nashida, Kanagawa (JP); and Naomasa Takahashi, Chiba (JP)
Assigned to SONY GROUP CORPORATION, Tokyo (JP)
Filed by SONY GROUP CORPORATION, Minato-ku (JP)
Filed on Feb. 17, 2021, as Appl. No. 17/177,933.
Claims priority of provisional application 63/089,413, filed on Oct. 8, 2020.
Claims priority of provisional application 63/090,773, filed on Oct. 13, 2020.
Prior Publication US 2022/0114379 A1, Apr. 14, 2022
Int. Cl. G06F 18/2413 (2023.01); B60K 35/00 (2006.01); B60K 35/10 (2024.01); B60W 30/09 (2012.01); G06N 20/00 (2019.01); G06V 10/44 (2022.01); G06V 10/75 (2022.01); G06V 20/58 (2022.01)
CPC G06F 18/2413 (2023.01) [B60K 35/00 (2013.01); B60W 30/09 (2013.01); G06N 20/00 (2019.01); G06V 10/443 (2022.01); G06V 10/751 (2022.01); G06V 20/58 (2022.01); B60K 35/10 (2024.01); B60K 2360/146 (2024.01); B60K 2360/334 (2024.01); B60W 2400/00 (2013.01); B60W 2420/403 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A system, comprising:
circuitry configured to:
acquire an output of image sensor circuitry;
determine a first object classification result based on feature points associated with one or more first objects in a first frame of the acquired output;
execute an event-camera signal processing operation on at least two frames of the acquired output to generate an event frame, wherein
the generated event frame comprises pixel-level movement information associated with one or more second objects;
determine a second object classification result based on the pixel-level movement information;
determine whether there is a difference between the determined first object classification result and the determined second object classification result;
determine one or more object classification tags based on the determination that there is no difference between the determined first object classification result and the determined second object classification result, wherein
the determined one or more object classification tags correspond to one or more objects included in at least the one or more first objects and the one or more second objects;
determine, based on the determination that there is the difference between the determined first object classification result and the determined second object classification result, an errored object from one of the one or more first objects or the one or more second objects, wherein
the determined errored object is one of misclassified, unclassified, detected but unclassified, mis-detected, or mis-detected and mis-classified;
execute a pattern matching operation for the determined errored object based on the determined difference, the determined first object classification result, and the determined second object classification result; and
classify the determined errored object into a first object class based on the execution of the pattern matching operation.