US 11,750,794 B2
Combining video-based and optic-based augmented reality in a near eye display
Nessi Benishti, Kfar Saba (IL); Nissan Elimelech, Beerotaim (IL); Stuart Wolf, Yokneam (IL); and Elazar Gerland, Hinanit (IL)
Assigned to Augmedics Ltd., Yokneam Illit (IL)
Filed by Augmedics Ltd., Yokneam Illit (IL)
Filed on May 22, 2019, as Appl. No. 16/419,023.
Application 16/419,023 is a continuation of application No. 16/159,740, filed on Oct. 15, 2018, granted, now 10,382,748.
Application 16/159,740 is a continuation of application No. 15/896,102, filed on Feb. 14, 2018, granted, now 10,134,166, issued on Nov. 20, 2018.
Application 15/896,102 is a continuation of application No. 15/127,423, granted, now 9,928,629, issued on Mar. 27, 2018, previously published as PCT/IB2016/051642, filed on Mar. 23, 2016.
Claims priority of application No. 1504935 (GB), filed on Mar. 24, 2015.
Prior Publication US 2019/0273916 A1, Sep. 5, 2019
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 3/01 (2006.01); G06T 19/00 (2011.01); H04N 13/383 (2018.01); G02B 27/01 (2006.01); H04N 13/344 (2018.01); G06V 10/60 (2022.01); G06V 40/18 (2022.01); H04N 5/33 (2023.01); H04N 7/18 (2006.01); G02B 27/00 (2006.01)
CPC H04N 13/383 (2018.05) [G02B 27/0172 (2013.01); G06F 3/013 (2013.01); G06F 3/017 (2013.01); G06T 19/006 (2013.01); G06V 10/60 (2022.01); G06V 40/193 (2022.01); H04N 5/332 (2013.01); H04N 7/181 (2013.01); H04N 13/344 (2018.05); G02B 27/0093 (2013.01); G02B 2027/0134 (2013.01); G02B 2027/0138 (2013.01); G02B 2027/0178 (2013.01); G02B 2027/0187 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system for conducting an augmented reality assisted medical procedure on a patient, comprising:
a surgical device for use by a medical professional in the procedure, the device comprising one or more device identifying elements;
a region of interest (ROI) marker comprising marker elements configured to define an ROI at a predefined position relative to the marker, positioned at a position selected by the medical professional so as to determine the ROI, wherein the ROI comprises a region within a surface, and wherein the marker elements are located in a perimeter of the surface;
one or more processors configured to access a database wherein are stored images; and
an augmented reality assembly comprising a retaining structure configured to be mounted on the medical professional, the assembly comprising:
a near eye display; and
a micro-projector configured to be operated by the one or more processors so as to present images on the near eye display; and
a capturing device configured to be operated by the one or more processors so as to image the ROI;
wherein:
the capturing device is configured to image the one or more device identifying elements and the marker elements of the ROI marker,
the one or more processors use the imaging of the identifying elements to track the surgical device,
the one or more processors use the predefined position and the imaging of the marker elements to track the ROI,
the one or more processors are configured to register a position of the tracked ROI with a given stored image from the database, and
the micro-projector is configured to project an image of the tracked surgical device with the registered given stored image on the near eye display.