US 11,947,590 B1
Systems and methods for contextualized visual search
Ria Chakraborty, Bangalore (IN); Madhur Popli, Ludhiana (IN); Rishi Kishore Verma, Bangalore (IN); and Pranesh Bhimarao Kaveri, Bangalore (IN)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Sep. 15, 2021, as Appl. No. 17/476,292.
Int. Cl. G06F 16/00 (2019.01); G06F 16/538 (2019.01); G06F 16/54 (2019.01); G06F 18/24 (2023.01); G06N 3/045 (2023.01); G06Q 30/0601 (2023.01)
CPC G06F 16/538 (2019.01) [G06F 16/54 (2019.01); G06F 18/24 (2023.01); G06N 3/045 (2023.01); G06Q 30/0641 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
one or more computer devices that implement a contextualized visual search (CVS) system, configured to:
receive, via a user interface, a query image and a search request to search for items that display a pictogram that includes the query image;
convert, using a first machine learning model, the query image into a query feature vector;
analyze, using a second machine learning model, target feature vectors of target images of a plurality of items, wherein the second machine learning model is trained to examine different sub-regions of a given target image to (a) determine whether individual ones of the sub-regions contain a view of the query image and (b) identify one or more locations of one or more views of the query image in the given target image;
identify, based at least in part on the analysis, a set of matching images from the target images that contain views of the query image and a set of matching items associated with the matching images; and
output search results via the user interface, wherein the search results indicate the matching items, the matching images, and locations of the views of the query image in the matching images.