CPC H04N 13/383 (2018.05) [G02B 27/0172 (2013.01); G06F 3/013 (2013.01); G06F 3/017 (2013.01); G06T 19/006 (2013.01); G06V 10/60 (2022.01); G06V 40/193 (2022.01); H04N 5/332 (2013.01); H04N 7/181 (2013.01); H04N 13/344 (2018.05); G02B 27/0093 (2013.01); G02B 2027/0134 (2013.01); G02B 2027/0138 (2013.01); G02B 2027/0178 (2013.01); G02B 2027/0187 (2013.01)] | 20 Claims |
1. A system for conducting an augmented reality assisted medical procedure on a patient, comprising:
a surgical device for use by a medical professional in the procedure, the device comprising one or more device identifying elements;
a region of interest (ROI) marker comprising marker elements configured to define an ROI at a predefined position relative to the marker, positioned at a position selected by the medical professional so as to determine the ROI, wherein the ROI comprises a region within a surface, and wherein the marker elements are located in a perimeter of the surface;
one or more processors configured to access a database wherein are stored images; and
an augmented reality assembly comprising a retaining structure configured to be mounted on the medical professional, the assembly comprising:
a near eye display; and
a micro-projector configured to be operated by the one or more processors so as to present images on the near eye display; and
a capturing device configured to be operated by the one or more processors so as to image the ROI;
wherein:
the capturing device is configured to image the one or more device identifying elements and the marker elements of the ROI marker,
the one or more processors use the imaging of the identifying elements to track the surgical device,
the one or more processors use the predefined position and the imaging of the marker elements to track the ROI,
the one or more processors are configured to register a position of the tracked ROI with a given stored image from the database, and
the micro-projector is configured to project an image of the tracked surgical device with the registered given stored image on the near eye display.
|