US 10,891,488 B2
System and method for neuromorphic visual activity classification based on foveated detection and contextual filtering
Deepak Khosla, Camarillo, CA (US); Ryan M. Uhlenbrock, Camarillo, CA (US); Huapeng Su, Los Angeles, CA (US); and Yang Chen, Westlake Village, CA (US)
Assigned to HRL Laboratories, LLC, Malibu, CA (US)
Filed by HRL Laboratories, LLC, Malibu, CA (US)
Filed on Jan. 14, 2019, as Appl. No. 16/247,157.
Application 16/247,157 is a continuation in part of application No. 15/947,032, filed on Apr. 6, 2018.
Application 15/947,032 is a continuation in part of application No. 15/883,822, filed on Jan. 30, 2018.
Claims priority of provisional application 62/642,959, filed on Mar. 14, 2018.
Claims priority of provisional application 62/516,217, filed on Jun. 7, 2017.
Claims priority of provisional application 62/479,204, filed on Mar. 30, 2017.
Prior Publication US 2019/0251358 A1, Aug. 15, 2019
Int. Cl. G06K 9/00 (2006.01); G06K 9/32 (2006.01); G06K 9/62 (2006.01); G06K 9/46 (2006.01); G06N 3/08 (2006.01)
CPC G06K 9/00744 (2013.01) [G06K 9/00718 (2013.01); G06K 9/00765 (2013.01); G06K 9/00771 (2013.01); G06K 9/3233 (2013.01); G06K 9/4628 (2013.01); G06K 9/6271 (2013.01); G06N 3/08 (2013.01)] 21 Claims
OG exemplary drawing
 
1. A system for visual activity classification, the system comprising:
one or more processors and a memory, the memory being a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions, the one or more processors perform operations of:
detecting a set of objects of interest in video data and determining an object classification for each object in the set of objects of interest, the set comprising at least one object of interest;
forming a corresponding activity track for each object in the set of objects of interest by tracking each object across frames, each activity track representing a position of an object in the video data across frames;
for each object of interest and using a feature extractor, determining a corresponding feature in the video data by performing feature extraction based on the corresponding activity track;
for each object of interest, based on the output of the feature extractor, determining a corresponding initial activity classification for each object of interest;
performing foveated object detection on a foveated region to detect one or more additional objects of interest in the foveated region, the foveated region being a region of a predetermined size surrounding each activity track;
appending the initial object detection and foveated object detection into a new detected-objects list; and
classifying a final activity of each activity track using the new detected-objects list and filtering the initial activity classification results using contextual logic.