US 11,915,305 B2
Identification of physical products for augmented reality experiences in a messaging system
Jean Luo, Los Angeles, CA (US); and Celia Nicole Mourkogiannis, Los Angeles, CA (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Mar. 18, 2021, as Appl. No. 17/249,923.
Claims priority of provisional application 63/003,395, filed on Apr. 1, 2020.
Prior Publication US 2021/0312533 A1, Oct. 7, 2021
Int. Cl. G06Q 30/06 (2023.01); G06F 16/583 (2019.01); G06K 7/14 (2006.01); G06Q 20/32 (2012.01); G06Q 30/0601 (2023.01)
CPC G06Q 30/0643 (2013.01) [G06F 16/5854 (2019.01); G06K 7/1413 (2013.01); G06Q 20/3267 (2020.05); G06Q 20/3274 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A method, comprising:
receiving, by an image data receiving module of a client device, image data including a representation of a physical item, wherein the physical item comprises a beauty product;
analyzing, by a product identification module accessing a shared memory of the client device, the image data to determine an object corresponding to the physical item;
extracting product metadata based on the determined object;
sending, by the client device to a product catalog system over the Internet, the product metadata to determine second product metadata associated with the product metadata, the product catalog system comprising a first processor for processing the product metadata and a first memory for storing information related to the second product metadata associated with the product metadata of the determined object, the information comprising at least a set of manufacturers of the determined object;
receiving, by the product catalog system, the second product metadata from a database server over a network;
receiving, by the client device from the product catalog system over the Internet, the second product metadata, the second product metadata including additional information related to the physical item, wherein the additional information includes information related to a set of similar products to the beauty product, the similar products being other products than the beauty product; and
causing display, at the client device, the additional information related to the physical item based at least in part on the second product metadata, wherein causing display the additional information related to the physical item comprises:
providing for display an interface at the client device, the interface comprising a plurality of media content based on the second product metadata, the plurality of media content provided by an external source separate from the client device, the interface further comprising a plurality of graphical representations of a plurality of similar products, each graphical representation from the plurality of graphical representations comprising an associated image showing a particular graphical representation of a particular product from the plurality of similar products, the plurality of graphical representations of the plurality of similar products being displayed below the plurality of media content, and
the interface further comprising a list of ingredients for at least one of the particular product from the plurality of similar products, the list of ingredients being displayed below the plurality of graphical representations of the plurality of similar products in the interface.