US 12,189,867 B2
Methods and systems for offloading pose processing to a mobile device for motion tracking on a hardware device without a camera
Ka Ho Lau, Kwai Chung (HK); On Loy Sung, Lai Chi Kok (HK); Wang Chiu Chan, Tseung Kwan (HK); Wang Fai Ng, Tuen Mun (HK); and Keng Fai Lee, Cupertino, CA (US)
Assigned to NEX Team Inc., San Jose, CA (US)
Filed by NEX Team Inc., San Jose, CA (US)
Filed on Mar. 12, 2024, as Appl. No. 18/602,061.
Claims priority of provisional application 63/452,704, filed on Mar. 17, 2023.
Prior Publication US 2024/0310921 A1, Sep. 19, 2024
Int. Cl. G06F 3/01 (2006.01); G06T 7/73 (2017.01)
CPC G06F 3/017 (2013.01) [G06T 7/73 (2017.01); G06T 2207/10016 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30196 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A non-transitory physical storage medium storing program code, the program code executable by a hardware processor, the hardware processor when executing the program code causing the hardware processor to execute a computer-implemented process for motion tracking on a hardware device without a camera and/or without a sufficiently powerful hardware processor, the process executable on a mobile device with a camera by offloading pose processing to the mobile device, the program code comprising code to:
pair the mobile device with the hardware device over a network connection, the network connection for transmitting pose data from the mobile device to the hardware device;
capture at least one image associated with a user;
perform a pose recognition algorithm on the mobile device, based on the captured image, to generate the pose data for performing a touchless control of an application on the hardware device, wherein the pose recognition algorithm further comprises program code to apply a convolutional neural network to the image to determine a feature map; program code to determine a keypoints map and an affinity-field map based on the feature map; and program code to determine the pose data of the user based on the keypoints map and the affinity-field map; and
transmit the pose data to the hardware device over the network connection.