US 11,995,899 B2
Pointer-based content recognition using a head-mounted device
Qinge Wu, Fremont, CA (US); Grant Yoshida, Sunnyvale, CA (US); Catherine Boulanger, Palo Alto, CA (US); Erik Hubert Dolly Goossens, San Francisco, CA (US); Cem Keskin, San Francisco, CA (US); Sofien Bouaziz, Los Gatos, CA (US); Jonathan James Taylor, San Francisco, CA (US); Nidhi Rathi, San Francisco, CA (US); and Seth Raphael, Portland, OR (US)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Apr. 29, 2021, as Appl. No. 17/302,291.
Prior Publication US 2022/0350997 A1, Nov. 3, 2022
Int. Cl. G06V 20/62 (2022.01); G02B 27/00 (2006.01); G02B 27/01 (2006.01); G06V 10/20 (2022.01); G06V 30/10 (2022.01)
CPC G06V 20/63 (2022.01) [G02B 27/0093 (2013.01); G02B 27/017 (2013.01); G06V 10/255 (2022.01); G02B 2027/0138 (2013.01); G02B 2027/014 (2013.01); G06V 30/10 (2022.01)] 17 Claims
OG exemplary drawing
 
1. A computer program product, the computer program product being tangibly embodied on a non-transitory computer-readable storage medium and comprising instructions that, when executed by at least one computing device, are configured to cause the at least one computing device to:
determine, at a head-mounted device (HMD), a request for recognizing at least one content item included within content framed within a display of the HMD;
initiate, in response to the request, a head-tracking process that maintains a coordinate system with respect to the content;
initiate, in response to the request, a pointer-tracking process that tracks a pointer that is visible together with the content within the display, the head-tracking process and the pointer-tracking process executing at least partially in parallel with one another;
capture, using the HMD, a first image of the content;
capture, using the HMD, a second image of the content while the head-tracking process and the pointer-tracking process are active, the second image including the pointer;
map a location of the pointer within the second image to a corresponding image location within the first image, using the coordinate system; and
provide the at least one content item from the corresponding image location while the head-tracking process and the pointer-tracking process are active.