US 12,340,477 B2
Contextual-based rendering of virtual avatars
Thomas Marshall Miller, IV, Los Angeles, CA (US); Josh Anon, Los Angeles, CA (US); Frank Alexander Hamilton, IV, Martinsburg, WV (US); Cole Parker Heiner, Vista, CA (US); Victor Ng-Thow-Hing, Los Altos, CA (US); Rodrigo Cano, Plantation, FL (US); Karen Stolzenberg, Venice, CA (US); Lorena Pazmino, Wilton Manors, FL (US); Gregory Minh Tran, Miami, FL (US); Stephane Antoine Joseph Imbert, Cary, NC (US); and Anthony Marinello, Cary, NC (US)
Assigned to MAGIC LEAP, INC., Plantation, FL (US)
Filed by Magic Leap, Inc., Plantation, FL (US)
Filed on May 19, 2023, as Appl. No. 18/320,904.
Application 18/320,904 is a division of application No. 17/833,508, filed on Jun. 6, 2022, granted, now 11,699,255.
Application 17/833,508 is a division of application No. 16/208,119, filed on Dec. 3, 2018, granted, now 11,380,039, issued on Jul. 5, 2022.
Claims priority of provisional application 62/598,905, filed on Dec. 14, 2017.
Prior Publication US 2023/0290031 A1, Sep. 14, 2023
Int. Cl. G06T 19/00 (2011.01); G06F 3/01 (2006.01); G06F 3/03 (2006.01); G06F 17/18 (2006.01); G06V 40/20 (2022.01); H04N 7/15 (2006.01); G06T 13/40 (2011.01)
CPC G06T 19/006 (2013.01) [G06F 3/011 (2013.01); G06F 3/012 (2013.01); G06F 3/013 (2013.01); G06F 3/017 (2013.01); G06F 3/0304 (2013.01); G06F 17/18 (2013.01); G06V 40/20 (2022.01); H04N 7/157 (2013.01); G06T 13/40 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A wearable device for determining an intent of an user interaction, the wearable device comprising:
a device sensor configured to acquire interaction data regarding interactions of a user with an environment of the user; and
a hardware processor programmed to:
determine an object of interest in the environment of the user, wherein to determine the object of interest the hardware processor:
extends a vector from the user towards each potential object of interest and a sight line vector that extends from the user outwards directly in front of the user;
assigns an interest weight value to each potential object of interest, wherein the interest weight of a potential object of interest dynamically changes to reflect changes in the environment; and
calculates an overall interest value for each potential object of interest, wherein the overall interest value for a potential object of interest is based at least partly on the interest weight value for the potential object of interest;
for each potential object of interest:
computing a dot product between the vector that extends from the user towards the potential object of interest and the sight line vector; and
multiplying the dot product by the interest weight value assigned to the potential object of interest to calculate the overall interest value for the potential object of interest; and
determining that the potential object of interest having a largest overall interest value is the object of interest;
identify a user interaction based on the interaction data acquired from the device sensor, wherein the user interaction includes at least one of an interaction with the object of interest in the environment of the user;
decompose the user interaction into at least one of: a world component or a local component, wherein the world component includes an action with reference to a world frame of the environment and the local component includes an action with reference to a local frame of a body of the user;
determine an intent associated with the user interaction based on contextual information, wherein the contextual information includes a past activity of the user; and
communicate at least one of: the intent, information associated with the world component, or information associated with the local component to another wearable device to cause the other wearable device to map the user interaction to an avatar of the user rendered by the other wearable device based on the intent.