US 12,461,590 B1
Identifying interactions with inventory locations
Abhay Doke, Framingham, MA (US); Gregory Donald Hager, Springfield, VA (US); and Harish Srinivasan, Redmond, WA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Mar. 28, 2022, as Appl. No. 17/706,168.
Int. Cl. G06F 3/01 (2006.01); G06Q 30/0201 (2023.01); G06T 7/70 (2017.01); G06V 10/25 (2022.01); G06V 10/74 (2022.01); G06V 20/50 (2022.01); G06V 20/52 (2022.01); G06V 20/58 (2022.01)
CPC G06F 3/013 (2013.01) [G06Q 30/0201 (2013.01); G06T 7/70 (2017.01); G06V 10/25 (2022.01); G06V 10/761 (2022.01); G06V 20/50 (2022.01); G06V 20/52 (2022.01); G06V 20/58 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
one or more processors; and
one or more computer-readable media storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
receiving video data generated by a camera associated with a shopping cart, the shopping cart being located within a facility that includes an inventory location storing one or more items, the video data corresponding to a first period of time;
determining that a portion of the video data represents a head of a user;
determining, based at least in part on the portion of the video data, gaze vectors indicating directions that the user looked during a second period of time that is within the first period of time;
determining, based at least in part on the gaze vectors, gaze points indicating focal points for a gaze of the user on a portion of the inventory location that the user was looking at during the second period of time;
generating, based at least in part on the gaze points, a heatmap associated with the inventory location for the user, the heatmap indicating that the user was looking at the portion of the inventory location for the second period of time;
determining a present location associated with the user within the facility;
identifying the inventory location based at least in part on the present location;
identifying a planogram associated with the inventory location;
determining, based at least in part on the planogram, an identifier of an item, of the one or more items, that is associated with the portion of the inventory location; and
generating metrics data representing at least the identifier of the item and the second period of time.