US 12,008,173 B2
Multi-sensor handle controller hybrid tracking method and device
Tao Wu, Shandong (CN)
Assigned to QINGDAO PICO TECHNOLOGY CO., LTD., Qingdao (CN)
Filed by QINGDAO PICO TECHNOLOGY CO., LTD., Shandong (CN)
Filed on Dec. 21, 2022, as Appl. No. 18/086,425.
Application 18/086,425 is a continuation of application No. PCT/CN2021/103544, filed on Jun. 30, 2021.
Claims priority of application No. 202010625647.9 (CN), filed on Jul. 1, 2020.
Prior Publication US 2023/0119687 A1, Apr. 20, 2023
Int. Cl. G06F 3/0346 (2013.01); G02B 27/01 (2006.01); G06F 3/01 (2006.01); G06T 7/277 (2017.01); G06T 7/73 (2017.01)
CPC G06F 3/0346 (2013.01) [G02B 27/0101 (2013.01); G02B 27/0172 (2013.01); G06F 3/011 (2013.01); G06T 7/277 (2017.01); G06T 7/73 (2017.01); G02B 2027/0138 (2013.01); G02B 2027/014 (2013.01); G06T 2207/20024 (2013.01); G06T 2207/20221 (2013.01)] 9 Claims
OG exemplary drawing
 
1. A multi-sensor handle controller hybrid tracking method, comprising:
acquiring tracking data of a handle controller, the tracking data comprising optical tracking data, electromagnetic tracking data, and inertial navigation data;
constructing a state transition model and an observation model of an extended Kalman filtering iteration strategy according to the tracking data, and performing an extended Kalman filtering fusion on the optical tracking data, the electromagnetic tracking data, and the inertial navigation data, wherein the constructing a state transition model and an observation model of an extended Kalman filtering iteration strategy according to the tracking data further comprises:
acquiring 6DoF data and velocity information of an optical pattern mark point on the handle controller relative to a first camera on a head mounted display in real time according to the optical tracking data,
converting 6DoF data with respect to other cameras into 6DoF data under a coordinate system with the first camera as an origin, by using calibration parameters among the cameras, which is denoted as Poseoptic,
converting electromagnetic 6DoF data of the handle controller into 6DoF data under the coordinate system with the first camera as the origin through calibration parameter of an electromagnetic signal receiver internally provided in the head mounted display and the first camera, which is denoted as PoseEM,
fusing the data Poseoptic and the data PoseEM into new pose data, which is denoted as PoseOEM, and
constructing the state transition model and the observation model of the extended Kalman filtering iteration strategy by using the new pose data; and
determining position and orientation information of the handle controller in a space according to the extended Kalman filtering iteration strategy.
 
9. A multi-sensor handle controller hybrid tracking device, wherein, a surface of a handle controller is provided with an optical pattern mark point, and the device comprises:
a plurality of cameras provided on a head mounted display and configured to track and shoot the optical pattern mark point;
an optical sensor configured to acquire optical tracking data of the optical pattern mark point;
an electromagnetic sensor configured to acquire electromagnetic tracking data of the handle controller;
an inertial navigation sensor configured to acquire inertial navigation data of the handle controller;
the head mounted display internally provided with a wireless transmission module and configured to receive the optical tracking data, the electromagnetic tracking data, and the inertial navigation data;
a model construction module configured to construct a state transition model and an observation model of an extended Kalman filtering iteration strategy and perform an extended Kalman filtering fusion on the optical tracking data, the electromagnetic tracking data, and the inertial navigation data, wherein constructing a state transition model and an observation model of an extended Kalman filtering iteration strategy according to the tracking data further comprises:
acquiring 6DoF data and velocity information of an optical pattern mark point on the handle controller relative to a first camera on a head mounted display in real time according to the optical tracking data,
converting 6DoF data with respect to other cameras into 6DoF data under a coordinate system with the first camera as an origin, by using calibration parameters among the cameras, which is denoted as Poseoptic,
converting electromagnetic 6DoF data of the handle controller into 6DoF data under the coordinate system with the first camera as the origin through calibration parameter of an electromagnetic signal receiver internally provided in the head mounted display and the first camera, which is denoted as PoseEM,
fusing the data and the data Poseoptic and the data PoseEM into the new pose data, which is denoted as PoseOEM, and
constructing the state transition model and the observation model of the extended Kalman filtering iteration strategy by using the new pose data; and
a tracking module configured to determine position and orientation information of the handle controller in a space according to the extended Kalman filtering iteration strategy.