US 11,935,503 B1
Semantic-based image mapping for a display
Sivalogeswaran Ratnasingam, San Jose, CA (US); Anselm Grundhoefer, Campbell, CA (US); and Ralf Habel, Cupertino, CA (US)
Assigned to APPLE INC., Cupertino, CA (US)
Filed by Apple Inc., Cupertino, CA (US)
Filed on Apr. 8, 2021, as Appl. No. 17/225,366.
Claims priority of provisional application 63/031,400, filed on May 28, 2020.
Int. Cl. G09G 5/10 (2006.01)
CPC G09G 5/10 (2013.01) [G09G 2320/0646 (2013.01); G09G 2354/00 (2013.01); G09G 2360/141 (2013.01); G09G 2360/144 (2013.01)] 24 Claims
OG exemplary drawing
 
1. A method comprising:
at an electronic device including one or more processors, a non-transitory memory, and a see-through display:
sensing a plurality of luminance values associated with ambient light from a physical environment, wherein the plurality of luminance values includes a first luminance value that quantifies a first amount of ambient light arriving at a first portion of the see-through display and a second luminance value that quantifies a second amount of ambient light arriving at a second portion of the see-through display, wherein the second amount of ambient light is less than the first amount of ambient light;
obtaining image data with a plurality of portions including a first portion that is to be displayed at the first portion of the see-through display and a second portion that is to be displayed at the second portion of the see-through display;
obtaining, via semantic segmentation, a plurality of semantic values respectively associated with the plurality of portions within the image data, wherein the plurality of semantic values includes a first semantic value identifying a first object represented by the first portion of the image data and a second semantic value identifying a second object represented by the second portion of the image data;
generating, based on the first luminance value and the first semantic value, a first mapped object corresponding to the first object; and
generating, based on the second luminance value and the second semantic value, a second mapped object corresponding to the second object, wherein the second mapped object has a greater luminance value than the second object in order to compensate for the second amount of light arriving at the second portion of the see-through display being less than the first amount of light arriving at the first portion of the see-through display.