US 11,704,716 B2
Identifying objects within an image from a user of an online system matching products identified to the online system by the user
Steve Morin, San Francisco, CA (US); Xuewei Ouyang, San Francisco, CA (US); Martin Mroz, San Francisco, CA (US); Anuj Madan, San Mateo, CA (US); and Aarati Vijay Soman, Menlo Park, CA (US)
Assigned to Meta Platforms, Inc., Menlo Park, CA (US)
Filed by Meta Platforms, Inc., Menlo Park, CA (US)
Filed on Feb. 20, 2020, as Appl. No. 16/796,635.
Claims priority of provisional application 62/937,067, filed on Nov. 18, 2019.
Prior Publication US 2021/0150611 A1, May 20, 2021
Int. Cl. G06Q 30/00 (2023.01); G06Q 30/0601 (2023.01); G06Q 30/0251 (2023.01)
CPC G06Q 30/0631 (2013.01) [G06Q 30/0253 (2013.01); G06Q 30/0254 (2013.01); G06Q 30/0256 (2013.01); G06Q 30/0641 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A method comprising:
obtaining, at an online system, a product catalog containing information identifying each of a plurality of products offered by a user of the online system, the obtained information including one or more pictures of different products;
receiving, at the online system, a request to post a content item from the user for presentation to other users by the online system, the content item including an image;
determining, by the online system, confidences of an object identified in the image matches each of a set of products offered by the user by applying an identification model to the identified object and to pictures of each product of the set, the identification model outputting a confidence of the object identified in the image matching a product of the set and the identification model trained using back propagation through a neural network comprising the identification model by the online system applying the identification model to characteristics of objects previously identified from images to which corresponding labels are applied indicating a product of the set matching a previously identified object and corresponding labels;
determining that the identified object has a confidence of matching a product of the set output by the identification model equaling or exceeding a threshold confidence value;
responsive to the determining:
generating, by the online system, a tag that includes metadata describing the product of the set for which the identified object has the confidence equaling or exceeding the threshold confidence value,
automatically including the tag in the content item by the online system, and
distributing the content item with the included tag as a post by the online system to one or more of the other users of the online system;
receiving, at the online system, a second request to post a second content item from the user for presentation to other users by the online system, the second content item including a second image;
determining, by the online system, a confidence that an object identified in the second image matches each of a set of products offered by the user by applying the identification model to the identified object and to pictures of each product of the set;
determining that the object identified in the second image has a confidence of matching each of a set of products output by the identification model being less than the threshold confidence value;
ranking products of the set of products based on the confidences of the identified object matching different products of the set;
selecting products of the set having at least a threshold position in the ranking;
displaying information identifying each of the selected products to the user in an order corresponding to positions of the selected objects in the ranking via an interface;
receiving a user selection of one of the selected products;
generating, by the online system, a tag that includes metadata describing the user-selected product of the set;
including the tag in the second content item by the online system, and
distributing the second content item with the included tag as a post by the online system to one or more of the other users of the online system.