US 12,449,909 B2
Systems, apparatus, and methods for gesture-based augmented reality, extended reality
Te-Won Lee, San Diego, CA (US); and Edwin Chongwoo Park, San Diego, CA (US)
Assigned to SoftEye, Inc., San Diego, CA (US)
Filed by SoftEye, Inc., San Diego, CA (US)
Filed on Aug. 7, 2023, as Appl. No. 18/366,374.
Application 18/366,374 is a continuation of application No. 18/061,257, filed on Dec. 2, 2022, granted, now 11,847,266.
Claims priority of provisional application 63/340,470, filed on May 11, 2022.
Claims priority of provisional application 63/285,453, filed on Dec. 2, 2021.
Prior Publication US 2024/0019940 A1, Jan. 18, 2024
Int. Cl. G06F 3/01 (2006.01); G06F 3/16 (2006.01); G06T 11/00 (2006.01); G06V 10/25 (2022.01); G06V 10/26 (2022.01); G06V 10/28 (2022.01); G06V 10/70 (2022.01); G06V 10/82 (2022.01); G06V 40/18 (2022.01); G06V 40/20 (2022.01); H04N 23/65 (2023.01)
CPC G06F 3/017 (2013.01) [G06F 3/013 (2013.01); G06F 3/167 (2013.01); G06T 11/00 (2013.01); G06V 10/25 (2022.01); G06V 10/26 (2022.01); G06V 10/28 (2022.01); G06V 10/70 (2022.01); G06V 10/82 (2022.01); G06V 40/18 (2022.01); G06V 40/20 (2022.01); G06V 40/28 (2022.01); H04N 23/651 (2023.01)] 21 Claims
OG exemplary drawing
 
1. A gesture-driven scalable processing apparatus, comprising:
a scalable processing subsystem comprising at least a machine learning processor;
a camera sensor logic in communication with the machine learning processor; and
a non-transitory computer-readable medium comprising a first set of instructions that when executed by the scalable processing subsystem, causes the scalable processing subsystem to:
train the machine learning processor to recognize a plurality of user-specific user interactions at a plurality of resolutions and a plurality of frame rates;
capture a first eye-tracking image at a first resolution and a first frame rate via the camera sensor logic;
recognize a first user-specific user interaction from the first eye-tracking image, where the first user-specific user interaction comprises a gaze fixation;
capture a second image at a second resolution and a second frame rate via the camera sensor logic;
recognize a second user-specific user interaction from the second image; and
determine a user gesture based on the first user-specific user interaction and the second user-specific user interaction.