US 11,887,405 B2
Determining features based on gestures and scale
Steven Dang, Plano, TX (US); Chih-Hsiang Chow, Coppell, TX (US); and Elizabeth Furlan, Plano, TX (US)
Assigned to Capital One Services, LLC, McLean, VA (US)
Filed by Capital One Services, LLC, McLean, VA (US)
Filed on Aug. 10, 2021, as Appl. No. 17/398,471.
Prior Publication US 2023/0051467 A1, Feb. 16, 2023
Int. Cl. G06K 9/62 (2022.01); G06V 40/20 (2022.01); G06V 10/40 (2022.01); G06V 40/16 (2022.01); G06F 18/21 (2023.01); G06V 10/25 (2022.01); G06V 10/778 (2022.01); G06V 10/82 (2022.01); G06V 40/10 (2022.01); G06V 40/18 (2022.01)
CPC G06V 40/28 (2022.01) [G06F 18/217 (2023.01); G06V 10/25 (2022.01); G06V 10/40 (2022.01); G06V 10/778 (2022.01); G06V 10/82 (2022.01); G06V 40/103 (2022.01); G06V 40/174 (2022.01); G06V 40/18 (2022.01); G06V 40/20 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
receiving, by a first device, image data comprising a person in an environment, wherein the environment comprises objects with each object comprising one or more features;
detecting the person in the image data;
determining a position of the person within the environment;
predicting, based on a determination of the position of the person and via a machine learning model trained to recognize a line-of-sight, a general area of focus of the person;
determining positions of the objects in the environment;
determining relative distances between the position of the person and the positions of the objects;
identifying, based on the predicted general area of focus, based on the positions of the objects in the environment, and based on the relative distances between the position of the person and the positions of the objects, a collection of one or more features;
recognizing, based on the image data and via a machine learning model trained to recognize gestures, one or more gestures, wherein each of the one or more gestures is associated with a timestamp;
determining, based on the recognized one or more gestures and their associated timestamps, a specific area of focus of the person;
determining, based on the collection of one or more features and based on the specific area of focus, one or more specific features;
determining, based on a first gesture of the one or more recognized gestures of the person and the one or more specific features, specific features of interest to the person;
determining, from the image data and based on a first timestamp associated with the first gesture, a facial expression of the person;
generating, based on the facial expression of the person, a score associated with the specific features of interest to the person; and
outputting, based on the score, an indication of a level of interest, of the person, in the specific features of interest.