US 11,720,223 B2
Virtual user input controls in a mixed reality environment
Adrian Kaehler, Los Angeles, CA (US); and John Adam Croston, Seattle, WA (US)
Assigned to MAGIC LEAP, INC., Plantation, FL (US)
Filed by Magic Leap, Inc., Plantation, FL (US)
Filed on Sep. 22, 2021, as Appl. No. 17/482,256.
Application 17/482,256 is a continuation of application No. 15/829,249, filed on Dec. 1, 2017, granted, now 11,150,777.
Claims priority of provisional application 62/430,308, filed on Dec. 5, 2016.
Claims priority of provisional application 62/430,279, filed on Dec. 5, 2016.
Prior Publication US 2022/0083198 A1, Mar. 17, 2022
Int. Cl. G06F 3/04815 (2022.01); G06N 3/02 (2006.01); G06F 3/01 (2006.01); G06T 19/00 (2011.01); G06N 20/00 (2019.01); G06V 10/46 (2022.01); G06V 20/20 (2022.01); G06V 40/10 (2022.01)
CPC G06F 3/04815 (2013.01) [G06F 3/017 (2013.01); G06N 3/02 (2013.01); G06N 20/00 (2019.01); G06T 19/006 (2013.01); G06V 10/464 (2022.01); G06V 20/20 (2022.01); G06V 40/107 (2022.01)] 18 Claims
OG exemplary drawing
 
1. A wearable system for providing a virtual remote control in a mixed reality environment, the wearable system comprising:
a mixed reality display for presenting a virtual image to a user;
an outward-facing imaging system configured to image an environment of the user; and
a hardware processor programmed to:
identify a physical remote associated with a parent device;
generate a virtual remote based at least partly on specifications of the physical remote, the virtual remote comprises a virtual button associated with an active volume of space;
cause the display to render the virtual remote;
cause the display to render a focus indicator, the focus indicator connecting the virtual remote with the parent device, such that the focus indicator identifies that the virtual remote is associated with the parent device by visually connecting the virtual remote rendered by the mixed reality display with at least one of (i) a real-world view of the parent device as seen by the user through the wearable system, or (ii) a real-world view of the physical remote as seen by the user through the wearable system:
identify an actuation of the virtual remote indicating an interaction with the parent device:
transmit an instruction to the parent device causing the parent device to perform a function as if the physical remote is actuated; and
cause the mixed reality display to generate a second focus indicator for the virtual button when movement of the virtual button passes a threshold condition comprising at least one of a threshold position, a threshold duration, or a threshold velocity.