US 12,299,629 B2
Systems and methods of defining and identifying product display areas on product display shelves
Sung H. Kim, Bentonville, AR (US); Asim S. Alvi, Kendall Park, NJ (US); Jody P. Cook, New York, NY (US); Kshitiz Gupta, Jersey City, NJ (US); Christopher R. Jescheling, Bentonville, AR (US); Matthew A. Jones, Bentonville, AR (US); Rahul S. Kala, Jersey City, NJ (US); Britto J. Michaelsamy, Bentonville, AR (US); Maxwell A. Yardley, Redmond, WA (US); Kumar Chandan, Newark, NJ (US); Jadran Sirotkovic, Dublin (IE); Mirko A. Arnold, Dublin (IE); Peter M. Terlecky, Yonkers, NY (US); and Tirui Wu, Tuam (IE)
Assigned to Walmart Apollo, LLC, Bentonville, AR (US)
Appl. No. 17/920,610
Filed by Walmart Apollo, LLC, Bentonville, AR (US)
PCT Filed Apr. 16, 2021, PCT No. PCT/US2021/027608
§ 371(c)(1), (2) Date Oct. 21, 2022,
PCT Pub. No. WO2021/216357, PCT Pub. Date Oct. 28, 2021.
Claims priority of provisional application 63/013,931, filed on Apr. 22, 2020.
Prior Publication US 2023/0252407 A1, Aug. 10, 2023
Int. Cl. G06Q 10/00 (2023.01); A47F 1/08 (2006.01); G06Q 10/087 (2023.01); G06V 10/74 (2022.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 20/60 (2022.01); G06V 30/10 (2022.01)
CPC G06Q 10/087 (2013.01) [A47F 1/08 (2013.01); G06V 10/74 (2022.01); G06V 10/765 (2022.01); G06V 10/82 (2022.01); G06V 20/60 (2022.01); G06V 30/10 (2022.01); G06V 2201/07 (2022.01)] 20 Claims
OG exemplary drawing
 
1. An inventory management system of a retail facility, the system comprising:
an image capture device having a field of view that includes a product display shelf of the retail facility having different products that are offered for sale to consumers at the retail facility arranged therein;
an electronic database that stores a planogram of product display shelves at the retail facility, the planogram indicating physical locations of the products arranged on the product display shelf; and
a computing device including a control circuit, the computing device being communicatively coupled to the image capture device and the electronic database, the control circuit being configured to:
obtain an image of a product display shelf captured by the image capture device;
process the image to extract image data from the image and perform object detection analysis to detect individual packages of each of the different products captured in the image, and to detect portions of the product display shelf adjacent to the individual packages of each of the different products captured in the image;
based on detection of the individual packages of each of the different products captured in the image and the portions of the product display shelf adjacent to the individual packages of each of the different products captured in the image, define a plurality of product display areas for the different products on the product display shelf captured in the image, wherein each of the product display areas includes the individual packages representing one of the different products;
process the image data to generate a plurality of virtual boundary lines in the image, wherein each of the virtual boundary lines surrounds one of the defined product display areas;
correlate the image having each of the virtual boundary lines surrounding one of the defined product display areas to the planogram stored in the electronic database in order to determine an identity of a product contained in the individual packages located in each one of the defined product display areas; and
assign product identifiers to the virtual boundary lines in the image, wherein a product identifier assigned to a virtual boundary line surrounding a product display area defined by the control circuit uniquely identifies the product contained in the individual packages located in the product display area surrounded by the virtual boundary line; and
update the electronic database to indicate an association between the boundary line with the product identifier assigned to the virtual boundary line.