US 12,437,170 B2
Subregion transformation for label decoding by an automated checkout system
Ganglu Wu, Shanghai (CN); Shiyuan Yang, Shanghai (CN); Xiao Zhou, Shanghai (CN); Qi Wang, Shanghai (CN); Qunwei Liu, Shanghai (CN); and Youming Luo, Shanghai (CN)
Assigned to Maplebear Inc., San Francisco, CA (US)
Filed by Maplebear Inc., San Francisco, CA (US)
Filed on Feb. 14, 2023, as Appl. No. 18/169,012.
Prior Publication US 2024/0202694 A1, Jun. 20, 2024
Int. Cl. G06K 7/14 (2006.01); G06Q 20/20 (2012.01); G06Q 30/0601 (2023.01); G06T 3/40 (2024.01); G06T 9/00 (2006.01); G06V 10/25 (2022.01)
CPC G06K 7/1413 (2013.01) [G06K 7/1443 (2013.01); G06Q 20/208 (2013.01); G06Q 30/0633 (2013.01); G06Q 30/0641 (2013.01); G06T 3/40 (2013.01); G06T 9/00 (2013.01); G06V 10/25 (2022.01); G06V 2201/07 (2022.01)] 15 Claims
OG exemplary drawing
 
1. A method comprising, at a computer system comprising a processor and a computer-readable medium:
receiving, by a processor of a shopping cart, an image captured by a camera coupled to the shopping cart, the image depicting a machine-readable label on an item;
identifying, by the processor of the shopping cart, a readable region of the image by applying a readability detection model to the image, wherein the readability detection model is a model that is trained to identify regions within images depicting machine-readable labels that are likely to be identifiable by a label decoding model, wherein the label decoding model is a machine-learning model that is trained to identify an item identifier encoded in machine-readable labels based on images of the machine-readable labels, wherein training the readability detection model comprises:
receiving a training image of a known machine-readable label, wherein the known machine-readable label is a machine-readable label encoding a known item identifier;
generating a test region on the training image, wherein the test region comprises a portion of the training image;
applying the label decoding model to the test region to determine whether the label detection model correctly identifies the known item identifier; and
responsive to the label detection model correctly identifying the known item identifier, generating a training example for the readability detection model based on the training image and the test region;
resizing, by the processor of the shopping cart, the readable region of the image to dimensions for input to the label decoding model;
identifying, by the processor of the shopping cart, an item identifier encoded in the machine-readable label depicted in the received image by applying the label decoding model to the resized readable region of the image, the item identifier corresponding to the item on which the machine-readable label is located; and
updating, by the processor of the shopping cart, a user interface displayed on a display of the shopping cart to include content related to the item associated with the item identifier encoded in the machine-readable label.