US 12,464,032 B2
System and method for visual identification of displayed applications in electronic communications
Sharon Hüffner, Berlin (DE); Richard B. Sutton, San Mateo, CA (US); and Devin H. Redmond, Encinitas, CA (US)
Assigned to THETA LAKE, INC., Santa Barbara, CA (US)
Filed by THETA LAKE, INC., Santa Barbara, CA (US)
Filed on Aug. 8, 2022, as Appl. No. 17/883,221.
Claims priority of provisional application 63/237,335, filed on Aug. 26, 2021.
Prior Publication US 2023/0069524 A1, Mar. 2, 2023
Int. Cl. H04L 65/401 (2022.01); G06F 21/50 (2013.01); G06V 20/40 (2022.01)
CPC H04L 65/4015 (2013.01) [G06F 21/50 (2013.01); G06V 20/46 (2022.01)] 17 Claims
OG exemplary drawing
 
1. A system comprising:
a memory;
a processor; and
a non-transitory, computer-readable storage medium storing a set of instructions executable by the processor, the set of instructions comprising instructions for:
interfacing with a plurality of heterogeneous online communication platforms that operate independently to provide services that allow users to work together digitally online;
ingesting video content from the plurality of heterogeneous online communication platforms, the video content comprising video of screen sharing sessions from the plurality of heterogeneous online communication platforms;
processing the video content from the plurality of heterogeneous online collaboration platforms to identify applications open in the screen sharing sessions, wherein the video content comprises a video of a screen share from a respective communication platform, the video of the screen share comprising a plurality of frames, and wherein processing the video content comprises:
analyzing frames of the video of the screen share to determine frame characterizing data of respective frames, wherein analyzing the frames of the video comprises extracting visual features from the respective frames, the frame characterizing data of the respective frames comprising the visual features extracted from the respective frames;
comparing the frame characterizing data of the respective frames to target application fingerprints for a plurality of target applications, the target application fingerprints comprising visual features extracted from images of the target applications;
determining, based on the comparing, that a particular target application, from the plurality of target applications, was open in the screen share, wherein determining that the particular target application was open comprises:
determining that the frame characterizing data of the respective frames is a match to a target application fingerprint of the particular target application, wherein the target application fingerprint for the particular target application comprises visual features extracted from application screen images associated with the particular target application and indicative of the particular target application being visible in the application screen images, the application screen images showing the particular target application in a plurality of states; and
based on the determining that the particular target application was open in the screen share, taking one or more remedial actions.