US 12,248,908 B2
Method for tracking placement of products on shelves in a store
Bradley Bogolea, South San Francisco, CA (US); Mirza Akbar Shah, South San Francisco, CA (US); Lorin Vandegrift, South San Francisco, CA (US); Luke Fraser, South San Francisco, CA (US); Jariullah Safi, South San Francisco, CA (US); and Jeffrey Gee, South San Francisco, CA (US)
Assigned to Simbe Robotics, Inc., South San Francisco, CA (US)
Filed by Simbe Robotics, Inc., South San Francisco, CA (US)
Filed on Apr. 22, 2022, as Appl. No. 17/727,044.
Application 17/727,044 is a continuation of application No. 16/578,406, filed on Sep. 23, 2019, granted, now 11,341,454.
Application 16/578,406 is a continuation of application No. 15/600,527, filed on May 19, 2017, granted, now 10,467,587, issued on Nov. 5, 2019.
Claims priority of provisional application 62/339,039, filed on May 19, 2016.
Prior Publication US 2022/0391825 A1, Dec. 8, 2022
Int. Cl. G06Q 10/087 (2023.01); G05D 1/00 (2024.01); G06F 18/21 (2023.01); G06F 18/22 (2023.01); G06V 20/52 (2022.01); H04N 1/00 (2006.01); H04N 23/66 (2023.01); H04N 23/698 (2023.01); H04N 23/80 (2023.01); H04N 25/615 (2023.01)
CPC G06Q 10/087 (2013.01) [G05D 1/0274 (2013.01); G05D 1/0282 (2013.01); G06F 18/21 (2023.01); G06F 18/22 (2023.01); G06V 20/52 (2022.01); H04N 1/00 (2013.01); H04N 23/66 (2023.01); H04N 23/698 (2023.01); H04N 23/80 (2023.01); H04N 25/615 (2023.01); G05D 1/0246 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
by a robotic system deployed in a store;
autonomously navigating along a first shelving structure within the store;
capturing a first image of a first shelving structure, within a field of view of an optical sensor arranged in the robotic system while occupying a first location at a first time; and
transmitting the first image to a computer system; and
by the computer system:
receiving the first image depicting the first shelving structure in the store from the robotic system at the first time;
identifying a first tag, arranged on the first shelving structure, depicted in a first region of the first image;
detecting a first set of features in the first region of the first image, the first set of features representing a first product descriptor;
accessing a first set of template features of a first product associated with the first product descriptor;
identifying a first slot, proximal the first tag, depicted in a second region of the first image;
detecting absence of the first product in the first slot based on absence of features analogous to the first set of template features in the second region of the first image;
accessing a first quantity of units of the first product assigned to the first slot by a product database; and
in response to detecting absence of the first product in the first slot, generating a first prompt to restock the second first slot with the first quantity of units of the first product.