US 12,229,333 B2
Technique for visualizing interactions with a technical device in an XR scene
Daniel Rinck, Forchheim (DE); and Aniol Serra Juhe, Nuremberg (DE)
Assigned to SIEMENS HEALTHINEERS AG, Forchheim (DE)
Filed by Siemens Healthineers AG, Forchheim (DE)
Filed on Nov. 1, 2023, as Appl. No. 18/499,770.
Claims priority of application No. 22205125 (EP), filed on Nov. 2, 2022.
Prior Publication US 2024/0143069 A1, May 2, 2024
Int. Cl. G06F 3/01 (2006.01); G06F 3/03 (2006.01)
CPC G06F 3/011 (2013.01) [G06F 3/0304 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A computer-implemented method for visualizing interactions in an extended reality (XR) scene, the computer-implemented method comprising:
receiving a first dataset, the first dataset representing the XR scene including at least a technical device;
displaying the XR scene on an XR headset or a head-mounted display (HMD);
providing a room for a user, wherein the user wears the XR headset or HMD to interact with the XR scene, wherein the XR scene is displayed on the XR headset or HMD, wherein the room includes a set of optical sensors, and wherein the set of optical sensors includes at least one optical sensor at a fixed location relative to the room;
detecting, via the set of optical sensors, optical sensor data of the user as a second dataset while the user is interacting in the room with the XR scene, wherein the XR scene is displayed on the XR headset or HMD; and
fusing the first dataset and the second dataset to generate a third dataset, wherein
the first dataset, the second dataset and the third dataset include a point cloud, and
the third dataset includes a fusion of point clouds of the first dataset and the second dataset.