US 11,893,207 B2
Generating a semantic construction of a physical setting
Mark Drummond, Palo Alto, CA (US); Bo Morgan, Emerald Hills, CA (US); and Siva Chandra Mouli Sivapurapu, Santa Clara, CA (US)
Assigned to Apple Inc., Cupertino, CA (US)
Filed by Apple Inc., Cupertino, CA (US)
Filed on Sep. 14, 2021, as Appl. No. 17/475,004.
Application 17/475,004 is a continuation of application No. PCT/US2020/028959, filed on Apr. 20, 2020.
Claims priority of provisional application 62/837,282, filed on Apr. 23, 2019.
Prior Publication US 2021/0407185 A1, Dec. 30, 2021
Int. Cl. G06F 3/04815 (2022.01); G06F 16/53 (2019.01); G06T 17/00 (2006.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 20/70 (2022.01); G06V 20/20 (2022.01); G06V 10/22 (2022.01); G06K 7/14 (2006.01)
CPC G06F 3/04815 (2013.01) [G06F 16/53 (2019.01); G06K 7/1413 (2013.01); G06T 17/00 (2013.01); G06V 10/225 (2022.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 20/20 (2022.01); G06V 20/70 (2022.01); G06T 2210/04 (2013.01)] 26 Claims
OG exemplary drawing
 
1. A method comprising:
at a device including a non-transitory memory and one or more processors coupled with the non-transitory memory:
obtaining environmental data corresponding to a physical environment;
determining, based on the environmental data, a bounding surface of the physical environment;
detecting a physical element located within the physical environment based on the environmental data;
determining a semantic label for the physical element based on at least a portion of the environmental data corresponding to the physical element;
generating a semantic construction of the physical environment based on the environmental data, wherein the semantic construction of the physical environment includes a representation of the bounding surface, a representation of the physical element and the semantic label for the physical element; and
providing the semantic construction of the physical environment to a virtual intelligent agent (VIA) that generates actions for a graphical object that represents the VIA.