US 11,676,085 B2
System for detecting and classifying consumer packaged goods
Joel Iventosch, Austin, TX (US); James E. Dutton, Spicewood, TX (US); and Sean D. Matthews, Pflugerville, TX (US)
Assigned to Pensa Systems, Inc., Austin, TX (US)
Filed by Pensa Systems, Inc., Austin, TX (US)
Filed on Mar. 30, 2022, as Appl. No. 17/709,190.
Application 17/106,091 is a division of application No. 16/443,790, filed on Jun. 17, 2019, granted, now 10,885,395, issued on Jan. 5, 2021.
Application 17/709,190 is a continuation of application No. 17/106,091, filed on Nov. 28, 2020, granted, now 11,314,992.
Claims priority of provisional application 62/686,051, filed on Jun. 17, 2018.
Prior Publication US 2022/0222485 A1, Jul. 14, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G06Q 10/08 (2023.01); G06Q 10/0631 (2023.01); G06T 7/73 (2017.01); G06T 11/60 (2006.01); G06T 11/20 (2006.01); G06N 3/02 (2006.01); G06N 5/02 (2023.01); G06Q 10/087 (2023.01); G06V 10/25 (2022.01); G06V 20/10 (2022.01); G06F 18/2431 (2023.01); G06F 18/241 (2023.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 10/44 (2022.01)
CPC G06Q 10/06315 (2013.01) [G06F 18/241 (2023.01); G06F 18/2431 (2023.01); G06N 3/02 (2013.01); G06N 5/02 (2013.01); G06Q 10/087 (2013.01); G06T 7/73 (2017.01); G06T 11/20 (2013.01); G06T 11/60 (2013.01); G06V 10/25 (2022.01); G06V 10/454 (2022.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 20/10 (2022.01); G06V 20/176 (2022.01); G06T 2207/20084 (2013.01); G06T 2210/12 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A system for identifying consumer packaged goods (CPGs), comprising: an imaging device mounted on a mobile platform:
a CPG detector which is equipped with a neural network and which
(a) utilizes the imaging device to obtain an image containing a plurality of CPGs,
(b) detects a set of CPG superclass features in the image, and
(c) generates a CPG feature map which maps the location of CPG features in the image,
wherein said neural network has been trained on a set of images of CPGs to recognize which of a set of superclasses a detected CPG belongs to;
a region proposal network which accepts the CPG feature map as input and which returns a first set of regions in the image where a CPG could be located;
a superclassifier which examines each region in the first set of regions to determine if the region contains an instance of a superclass of a CPG, and which outputs a second set of regions containing a CPG; and
a superclass proposal layer which operates on each member of the second set of regions and returns a set of superclass proposals for each region;
wherein said imaging device is mounted on a drone;
wherein said region proposal network, said superclassifier, and said superclass proposal layer are defined in an operating system running in a tangible, non-transient memory medium installed on said drone.