US 11,989,772 B2
System and method for extracting product information from low resolution photos for updating product master
Vijay Gabale, Bengaluru (IN); Anand Prabhu Subramanian, Bengaluru (IN); and Vijaykumar Kannan, Bengaluru (IN)
Assigned to INFILECT TECHNOLOGIES PRIVATE LIMITED, Bengaluru (IN)
Appl. No. 17/623,980
Filed by INFILECT TECHNOLOGIES PRIVATE LIMITED, Bengaluru (IN)
PCT Filed Jul. 5, 2020, PCT No. PCT/IN2020/050588
§ 371(c)(1), (2) Date Dec. 30, 2021,
PCT Pub. No. WO2021/005621, PCT Pub. Date Jan. 14, 2021.
Claims priority of application No. 201941027078 (IN), filed on Jul. 5, 2019.
Prior Publication US 2022/0358571 A1, Nov. 10, 2022
Int. Cl. G06Q 30/0601 (2023.01); G06Q 20/20 (2012.01); G06V 10/44 (2022.01); G06V 10/82 (2022.01); G06V 20/52 (2022.01)
CPC G06Q 30/0643 (2013.01) [G06Q 20/20 (2013.01); G06V 10/454 (2022.01); G06V 10/82 (2022.01); G06V 20/52 (2022.01)] 16 Claims
OG exemplary drawing
 
1. A system for automatically extracting a plurality of product information of a plurality of assets from a plurality of low resolution media contents taken in a complex environment (104) and automatically updating a global product-master database (110) with the extracted product information, the system comprising:
an image capturing device (102) that captures a plurality of media contents of the plurality of assets in the environment (104);
a product information extraction system (106) that extracts the product information from the plurality of media contents;
a point of sale device (108) that comprises an in-built wireless transmitter and a receiver for updating a local point-of-sale product-master; and
the global product-master database (110) that is communicated with a global product-master server to obtain updated product information, wherein the product information extraction system (106) comprises:
a memory (200) that stores a database (202) and a set of instructions; and
a device processor (201) that executes the set of instructions and is configured to:
receive the media contents captured by the image capturing device (102), wherein the media contents comprise at least one of an image of an asset, a video of an asset, a shelf brand display, a point of sale brand display, a digital advertisement display or an image, a video or a three-dimensional model of at least one of a physical retail store environment, a digital retail store environment, a virtual reality store environment, a social media environment or a web page environment;
generate a database (202) of media contents associated with the environment (104);
determine the resolution of the media contents and identify the low resolution media contents, wherein the low resolution media contents are identified based on size and dimensions of the media contents;
parse the low resolution media contents, using a system of cascading deep neural networks to generate a product information, wherein the deep neural network system comprises:
(a) a first deep neural network that is configured to receive the low resolution media contents as input and provide super-resolution media contents of the low resolution media contents;
(b) a second neural network that is configured to receive the super-resolution media contents of the low resolution media contents from the first neural network and identifies a product present at a brand-form level by generating a plurality of bounding boxes and a brand-form level names for each of the boxes;
(c) a third neural network that is configured to receive a plurality of crop media contents of each of the bounding boxes produced at the second neural network as input and produces a plurality of high resolution pack-shot media contents by improving the resolution of the crop media contents, wherein the crop media contents are created by cropping each of the bounding boxes generated at second neural network; and
(d) a fourth neural network that is configured to receive the plurality of high resolution pack-shots produced at the third neural network as input and classifies the media contents into finest-level classes of product;
communicate with a point of sale device (108) and transfer the product information obtained from the deep neural network system, over a secure peer-to-peer protocol, to the point of sale terminal, which further updates a local point-of-sale product-master; and
establish a connection between the product information extraction system (106) and a global product-master server to transmit the extracted product information to the server to update a global product-master database (110).