US 12,475,705 B2
Spatial-temporal anomaly and event detection using night vision sensors
Subhodev Das, Princeton, NJ (US); Ajay Divakaran, Monmouth Junction, NJ (US); Ali Chaudhry, Princeton Junction, NJ (US); Julia Kruk, Rego Park, NY (US); and Bo Dong, Chatham, NJ (US)
Assigned to SRI International, Menlo Park, CA (US)
Filed by SRI International, Menlo Park, CA (US)
Filed on Jun. 7, 2023, as Appl. No. 18/331,007.
Claims priority of provisional application 63/349,854, filed on Jun. 7, 2022.
Prior Publication US 2024/0212350 A1, Jun. 27, 2024
Int. Cl. G06V 20/40 (2022.01); G06V 10/44 (2022.01); H04N 23/21 (2023.01)
CPC G06V 20/44 (2022.01) [G06V 10/44 (2022.01); H04N 23/21 (2023.01)] 16 Claims
OG exemplary drawing
 
1. A method comprising:
processing, by a computing system, using a first machine learning model, first content of multimodal data to generate a first modality feature vector representative of the first content, wherein the multimodal data comprises multidimensional spectral data comprising multi-resolution data in both space and time, and wherein the first content has a first modality and the first modality feature vector has the first modality;
processing, by the computing system, using the first machine learning model, second content of the multimodal data to generate a second modality feature vector representative of the second content, wherein the second content has a second modality and the second modality feature vector has the second modality, wherein the first modality is different than the second modality;
processing, by the computing system, using a second machine learning model, the first modality feature vector and the second modality feature vector to generate event data comprising at least one of an event or an activity of interest; and
processing, by the computing system, the event data to generate anomaly data indicative of detected anomalies in the multimodal data.