US 12,437,263 B2
Systems and methods of monitoring location labels of product storage structures of a product storage facility
Chongrui Zhao, San Mateo, CA (US); Shanthi Narayanan, Parrish, FL (US); and Tracy E. Benson, Campbell, CA (US)
Assigned to Walmart Apollo, LLC, Bentonville, AR (US)
Filed by Walmart Apollo, LLC, Bentonville, AR (US)
Filed on May 30, 2023, as Appl. No. 18/203,499.
Prior Publication US 2024/0403819 A1, Dec. 5, 2024
Int. Cl. G06Q 10/00 (2023.01); G06Q 10/0631 (2023.01); G06Q 10/087 (2023.01); G06V 20/00 (2022.01); G06V 20/50 (2022.01); G06V 20/62 (2022.01); G06V 30/00 (2022.01); G06V 30/10 (2022.01); G06K 7/14 (2006.01); G06K 15/02 (2006.01)
CPC G06Q 10/087 (2013.01) [G06Q 10/06316 (2013.01); G06V 20/50 (2022.01); G06V 20/63 (2022.01); G06V 30/10 (2022.01); G06K 7/1413 (2013.01); G06K 15/024 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system for monitoring location labels on product storage structures of a product storage facility, the system comprising:
an image capture device having a field of view that includes a product storage structure at the product storage facility configured to have products arranged thereon, wherein the image capture device is configured to capture at least one image of the product storage structure, and wherein the image capture device comprises a motorized robotic unit that autonomously moves around the product storage facility;
a computing device including a control circuit, the computing device communicatively coupled to the image capture device, the control circuit being configured to:
determine movement instructions for the motorized robotic unit;
autonomously control the motorized robotic unit by sending the movement instructions, wherein the motorized robotic unit enables the image capture device to capture the at least one image of the product storage structure in sequence;
analyze the at least one image of the product storage structure captured by the image capture device to detect at least one location label located on the product storage structure, wherein a trained machine learning model is used to detect the at least one location label; and
based on detection that the at least one location label located on the product storage structure is at least one of missing, incomplete, damaged, deformed, and at least partially obstructed, generate a location label alert indicating that the at least one location label requires a location label check by a worker at the product storage facility; and
a mobile application executable on a user device of the worker at the product storage facility, the mobile application, when executed, is configured to cause a user interface to be displayed to the worker on a display of the user device, wherein the user interface lists at least one location label alert generated by the control circuit of the computing device, wherein the user interface permits the worker to:
view a location of the product storage structure associated with the location label alert;
input an inspection result of a physical inspection of the at least one location label of the product storage structure, wherein the inspection result of the physical inspection is used to retrain the trained machine learning model;
generate a replacement label for the product storage structure associated with the location label alert and output signaling to cause the replacement label to be printed; and
scan the replacement label to verify that the worker affixed the replacement label to the product storage structure associated with the location label alert.