| CPC H04N 13/344 (2018.05) [A61B 34/20 (2016.02); H04N 5/33 (2013.01); H04N 13/239 (2018.05); H04N 23/12 (2023.01); A61B 2017/00991 (2013.01); A61B 2034/2057 (2016.02); A61B 2034/2072 (2016.02); A61B 34/30 (2016.02); A61B 2090/365 (2016.02); A61B 2090/3764 (2016.02); A61B 2090/3945 (2016.02); A61B 2090/502 (2016.02)] | 20 Claims |

|
1. A head-mounted extended reality (XR) display device comprising:
a frame configured to be worn by a user's head;
a rigid mounting element coupled to the frame;
a camera coupled to the rigid mounting element;
wherein the camera is configured to capture stereoscopic visible light images within a field of view of the user when the user is wearing the frame, and
a tracking system adapted to determine a pose of dynamic reference arrays attached to a patient and to surgical instruments based on stereoscopic visible light images received through the camera, the tracking system further adapted to determine hand gesture based commands based on stereoscopic NIR images received through the camera.
|