US 11,656,681 B2
System and method for determining user interactions with visual content presented in a mixed reality environment
Spandana Govindgari, East Windsor Township, NJ (US)
Assigned to HYPEAR, INC., San Jose, CA (US)
Filed by HYPE AR, Inc., San Jose, CA (US)
Filed on Aug. 31, 2020, as Appl. No. 17/7,429.
Prior Publication US 2022/0066550 A1, Mar. 3, 2022
Int. Cl. G06F 3/01 (2006.01); G06Q 30/02 (2012.01); G06V 40/20 (2022.01); G06V 40/16 (2022.01); G06Q 30/0241 (2023.01); G06Q 30/0251 (2023.01)
CPC G06F 3/013 (2013.01) [G06F 3/017 (2013.01); G06Q 30/0267 (2013.01); G06Q 30/0277 (2013.01); G06V 40/174 (2022.01); G06V 40/20 (2022.01)] 17 Claims
OG exemplary drawing
 
1. A system for determining user interactions with visual content presented in a mixed reality environment, the system comprising:
at least one network communication interface;
at least one non-transitory storage device; and
at least one processing device coupled to the at least one non-transitory storage device and the at least one network communication interface, wherein the at least one processing device is configured to:
generate visual content comprising context associated with an advertisement, wherein the visual content is application specific visual content generated based on an identified type of an application used by a user to access a mixed reality environment, wherein the application specific visual content matches an environment of the application used by the user to access the mixed reality environment;
display the visual content associated with the advertisement in the mixed reality environment of the user, via a user device of the user;
in response to displaying the visual content, perform at least one of one or more actions of:
continuously monitoring gaze of the user via the user device;
continuously monitoring one or more gestures of the user, via the user device; and
continuously monitoring one or more expressions of the user, via the user device;
determine that the user is interacting with the visual content based on at least one of the gaze, the one or more gestures, and the one or more expressions of the user;
generate additional visual content in response to determining that the user is interacting with the visual content, wherein the additional visual content is associated with the visual content and the advertisement; and
display the additional visual content in the mixed reality environment of the user, via the user device of the user.