US 11,854,148 B2
Virtual content display opportunity in mixed reality
Kausik Ghatak, Bengaluru (IN)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Nov. 16, 2018, as Appl. No. 16/193,428.
Prior Publication US 2020/0160602 A1, May 21, 2020
Int. Cl. G06T 19/00 (2011.01); G06F 3/01 (2006.01); G06Q 30/0241 (2023.01); G02B 27/01 (2006.01); G06F 3/0482 (2013.01); G06F 3/04845 (2022.01); G06V 20/10 (2022.01); H04L 67/01 (2022.01); G06T 19/20 (2011.01)
CPC G06T 19/006 (2013.01) [G02B 27/0172 (2013.01); G06F 3/013 (2013.01); G06F 3/017 (2013.01); G06Q 30/0277 (2013.01); G02B 27/017 (2013.01); G02B 27/0179 (2013.01); G02B 2027/014 (2013.01); G02B 2027/0138 (2013.01); G06F 3/012 (2013.01); G06F 3/0482 (2013.01); G06F 3/04845 (2013.01); G06T 19/20 (2013.01); G06V 20/10 (2022.01); H04L 67/01 (2022.05)] 13 Claims
OG exemplary drawing
 
1. A head-mounted display device, comprising:
an at least partially see-through display;
one or more input devices including:
one or more image sensors configured to collect imaging data of a physical environment; and
an inward-facing image sensor configured to collect inward-facing imaging data of a user;
one or more communication devices; and
a processor configured to:
output for display on the display a mixed reality experience including one or more virtual objects superimposed upon the physical environment;
transmit the imaging data to a server computing device via the one or more communication devices;
receive, from the server computing device, an identification of a virtual content item display opportunity, wherein the identification of the virtual content item display opportunity includes a determination that the mixed reality experience and one or more physical features identified from the imaging data satisfy spatial and temporal constraints;
transmit a request for a virtual content item to the server computing device based on the identification of the virtual content item display opportunity, wherein the request includes a dynamically positioned content item location in the physical environment indicated by the virtual content item display opportunity;
in response to transmitting the request, receive a response from the server computing device, the response including the virtual content item; and
output the virtual content item for display on the display at the selected dynamically positioned content item location;
determine, based at least in part on the inward-facing imaging data, that the user is gazing at the virtual content item and that the virtual content item is occluded by a physical object or a virtual object; and
transmit an indication of a user interaction to the server computing device via the one or more communication devices, wherein:
the user interaction includes gazing at the virtual content item; and
the user interaction further indicates that the virtual content item is occluded.