US 12,380,484 B1
Contextually relevant user-based product recommendations based on scene information
Yash Chaturvedi, Issaquah, WA (US); Mohamed Kamal Omar, Seattle, WA (US); Alexander Ratnikov, Redmond, WA (US); Ahmed Aly Saad Ahmed, Bothell, WA (US); Steven James Cox, Mill Creek, WA (US); Prasanth Saraswatula, Bellevue, WA (US); and Jingxiang Chen, Bellevue, WA (US)
Assigned to AMAZON TECHNOLOGIES, INC., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Jul. 26, 2021, as Appl. No. 17/385,406.
Int. Cl. G06Q 30/00 (2023.01); G06N 20/00 (2019.01); G06Q 30/0601 (2023.01)
CPC G06Q 30/0631 (2013.01) [G06N 20/00 (2019.01)] 16 Claims
OG exemplary drawing
 
1. A system, comprising:
one or more processors; and
memory storing executable instructions that, as a result of execution by the one or more processors, cause the system to:
parse a scene in a video content to identify a first object in multiple frames of the scene, the multiple frames including a first frame;
determine first information associated with the first object;
determine, using a second machine-learning model and based on the first information, similarity scores between the first object and one or more products of an online purchasing system, wherein the second machine-learning model is trained on a database of product images including images for the one or more products;
detect, in association with the first frame, performance of a first computer-based action by a user;
determine, based on contextual data of the user, one or more affinity scores for the user, wherein a first affinity score associated with a first product of the one or more products indicates a likelihood the user is to perform a second computer-based action in association with the first product;
select the first product based on the one or more affinity scores;
encode the video content to include an annotation associated with the first object, the annotation including third information associated with the first product; and
transmit the encoded video content to a user device.