US 12,067,527 B2
System and method for identifying misplaced products in a shelf management system
Marios Savvides, Wexford, PA (US); Sreena Nallamothu, Pittsburgh, PA (US); Magesh Kannan, Pittsburgh, PA (US); Uzair Ahmed, Pittsburgh, PA (US); Ran Tao, Pittsburgh, PA (US); and Yutong Zheng, Pittsburgh, PA (US)
Assigned to Carnegie Mellon University, Pittsburgh, PA (US)
Filed by CARNEGIE MELLON UNIVERSITY, Pittsburgh, PA (US)
Filed on Aug. 12, 2021, as Appl. No. 17/400,996.
Claims priority of provisional application 63/069,455, filed on Aug. 24, 2020.
Claims priority of provisional application 63/065,912, filed on Aug. 14, 2020.
Claims priority of provisional application 63/064,670, filed on Aug. 12, 2020.
Prior Publication US 2022/0051177 A1, Feb. 17, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G06Q 10/087 (2023.01); G06F 16/583 (2019.01); G06F 18/2113 (2023.01); G06F 18/214 (2023.01); G06F 18/28 (2023.01); G06V 10/40 (2022.01); G06V 10/75 (2022.01); G06V 20/52 (2022.01); H04N 7/18 (2006.01)
CPC G06Q 10/087 (2013.01) [G06F 16/5846 (2019.01); G06F 18/2113 (2023.01); G06F 18/214 (2023.01); G06F 18/28 (2023.01); G06V 10/40 (2022.01); G06V 10/751 (2022.01); G06V 20/52 (2022.01); H04N 7/18 (2013.01)] 15 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
obtaining an image containing multiple objects using a camera;
obtaining identifying information associated with a region of interest in the image;
extracting an object image from the image, the object image containing an individual object located in the region of interest;
extracting features from the object image using a trained feature extractor;
determining a best-fit match between features extracted from the object image and features associated with objects in an object library;
receiving an identifier associated with the best-fit match;
determining that the best-fit match identifier does not match the identifying information associated with the region of interest; and
indicating that the object in the object image is not associated with the other objects in the region of interest;
wherein the method is implemented in software executing on a processor; and
wherein the trained feature extractor is a deep neural network trained to on a dataset comprising multiple views of each object and associated identifying information of each object.