US 11,961,291 B2
Visual search for electronic devices
Lukas A. Bossard, Cupertino, CA (US); Matthias Dantone, Cupertino, CA (US); Pulah J. Shah, Cupertino, CA (US); Wentao Wu, Cupertino, CA (US); and Yang Zhao, Cupertino, CA (US)
Assigned to Apple Inc., Cupertino, CA (US)
Filed by Apple Inc., Cupertino, CA (US)
Filed on Nov. 16, 2021, as Appl. No. 17/528,157.
Claims priority of provisional application 63/176,878, filed on Apr. 19, 2021.
Prior Publication US 2022/0335242 A1, Oct. 20, 2022
Int. Cl. G06V 20/20 (2022.01); G06F 18/40 (2023.01); G06N 20/00 (2019.01); G06Q 30/0601 (2023.01); G06T 11/00 (2006.01); G06V 10/25 (2022.01); H04N 23/617 (2023.01); H04N 23/63 (2023.01); H04N 23/80 (2023.01)
CPC G06V 20/20 (2022.01) [G06F 18/40 (2023.01); G06N 20/00 (2019.01); G06Q 30/0625 (2013.01); G06T 11/00 (2013.01); G06V 10/25 (2022.01); H04N 23/617 (2023.01); H04N 23/635 (2023.01); H04N 23/80 (2023.01)] 20 Claims
OG exemplary drawing
 
1. A method, comprising:
obtaining, by a system process of a computing device, an image being displayed by an application at the computing device;
identifying, by the system process and responsive to providing the image to one or more machine learning models at the computing device, an element of interest in the image, the identifying further comprising:
identifying a plurality of elements in the image using the one or more machine learning models;
ranking the plurality of elements based on a ranking algorithm; and
based on the ranking, determining the element of interest, from amongst the plurality of elements; and
displaying a visual indicator for the element of interest overlaid on the image at a location corresponding to the element of interest, wherein the visual indicator is selectable to cause a display of information associated with the element of interest.