US 12,450,949 B2
Posture and motion monitoring using mobile devices
Aditya Sarathy, Santa Clara, CA (US); Rene F. Aguirre Ramos, Poway, CA (US); Umamahesh Srinivas, Milpitas, CA (US); and Yongyang Nie, New York, NY (US)
Assigned to Apple Inc., Cupertino, CA (US)
Filed by Apple Inc., Cupertino, CA (US)
Filed on Sep. 22, 2022, as Appl. No. 17/950,664.
Claims priority of provisional application 63/248,357, filed on Sep. 24, 2021.
Prior Publication US 2023/0096949 A1, Mar. 30, 2023
Int. Cl. G06V 10/00 (2022.01); G06T 7/30 (2017.01); G06T 7/50 (2017.01); G06T 7/73 (2017.01); G06V 40/20 (2022.01)
CPC G06V 40/23 (2022.01) [G06T 7/30 (2017.01); G06T 7/50 (2017.01); G06T 7/75 (2017.01)] 16 Claims
OG exemplary drawing
 
1. A method comprising:
obtaining, with at least one processor, motion data obtained from at least one motion sensor worn by a user;
obtaining, with the at least one processor, at least one frame of skeletal data of the user based on at least one of camera data or depth data;
calibrating, with the at least one processor, the motion data and skeletal data to determine a calibration offset, wherein determining the calibration offset includes:
determining a calibration error based on a difference between an estimated joint position and velocity from the skeletal data and a previously estimated joint position and velocity from the skeletal data;
updating the calibration offset based on the calibration error;
synchronizing the motion data and skeletal data;
generating the estimated joint position and velocity by:
aligning reference frames of the synchronized motion data and the synchronized skeletal data based on the calibration offset;
estimating the joint position and velocity based on the aligned reference frames of the motion data and skeletal data; and
classifying, by a machine learning model, an estimated body pose of the user based on the estimated joint position and velocity.