US 12,437,258 B2
System and method for identifying products in a shelf management system
Marios Savvides, Wexford, PA (US); Uzair Ahmed, Pittsburgh, PA (US); Sreena Nallamothu, Pittsburgh, PA (US); Magesh Kannan, Pittsburgh, PA (US); and Abhishek Das, Pittsburgh, PA (US)
Assigned to CARNEGIE MELLON UNIVERSITY, Pittsburgh, PA (US)
Filed by CARNEGIE MELLON UNIVERSITY, Pittsburgh, PA (US)
Filed on Oct. 20, 2021, as Appl. No. 17/506,115.
Application 17/506,115 is a continuation in part of application No. 17/400,996, filed on Aug. 12, 2021, granted, now 12,067,527.
Claims priority of provisional application 63/107,863, filed on Oct. 30, 2020.
Claims priority of provisional application 63/069,455, filed on Aug. 24, 2020.
Claims priority of provisional application 63/065,912, filed on Aug. 14, 2020.
Claims priority of provisional application 63/064,670, filed on Aug. 12, 2020.
Prior Publication US 2022/0051179 A1, Feb. 17, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G06Q 10/087 (2023.01); G06F 16/583 (2019.01); G06F 18/22 (2023.01); G06V 10/75 (2022.01); H04N 5/28 (2006.01); H04N 23/90 (2023.01)
CPC G06Q 10/087 (2013.01) [G06F 16/583 (2019.01); G06F 18/22 (2023.01); G06V 10/751 (2022.01); H04N 5/28 (2013.01); H04N 23/90 (2023.01)] 14 Claims
OG exemplary drawing
 
1. A system comprising:
a processor;
a camera;
a feature gallery for storing, for a plurality of objects, one or more multi-dimensional feature vectors extracted from images of the objects and one or more object identifiers associated with the objects; and
software executing on the processor, the software comprising;
a feature extractor for extracting feature vectors from a probe image collected using the camera;
a matching module for matching a product depicted in the probe image with a product stored in the feature gallery;
wherein the matching module calculates a distance between the one or more feature vectors extracted from the probe image and one or more feature vectors in the feature gallery by choosing, as a matching object, an object from the feature gallery having one or more feature vectors with the smallest distances from the one or more feature vectors extracted from the probe image; and
wherein the feature extractor is a multi-layered machine learning model comprising a plurality of trained convolutional neural networks, each convolutional neural network trained to extract a different type of feature vector characterizing a different feature from the probe image.