US 11,717,376 B2
System and method for dynamic validation, correction of registration misalignment for surgical navigation between the real and virtual images
Gal Sela, Toronto (CA); Cameron Piron, Toronto (CA); Michael Wood, Toronto (CA); Joshua Richmond, Toronto (CA); Murugathas Yuwaraj, Markham (CA); Stephen McFadyen, Toronto (CA); Monroe M. Thomas, Toronto (CA); Wes Hodges, London (CA); Simon Alexander, Toronto (CA); and David Gallop, Toronto (CA)
Assigned to Synaptive Medical Inc., Toronto (CA)
Filed by SYNAPTIVE MEDICAL INC., Toronto (CA)
Filed on Sep. 9, 2020, as Appl. No. 17/15,186.
Application 17/015,186 is a continuation of application No. 14/775,759, granted, now 10,799,316, previously published as PCT/CA2014/050266, filed on Mar. 14, 2014.
Claims priority of provisional application 61/924,993, filed on Jan. 8, 2014.
Claims priority of provisional application 61/818,280, filed on May 1, 2013.
Claims priority of provisional application 61/799,735, filed on Mar. 15, 2013.
Claims priority of provisional application 61/800,155, filed on Mar. 15, 2013.
Claims priority of provisional application 61/801,530, filed on Mar. 15, 2013.
Prior Publication US 2020/0405433 A1, Dec. 31, 2020
Int. Cl. A61B 90/00 (2016.01); A61B 34/20 (2016.01); A61B 34/10 (2016.01); G06T 7/33 (2017.01); A61B 90/50 (2016.01); G06T 7/246 (2017.01); G06T 7/73 (2017.01)
CPC A61B 90/39 (2016.02) [A61B 34/10 (2016.02); A61B 34/20 (2016.02); G06T 7/248 (2017.01); G06T 7/33 (2017.01); G06T 7/337 (2017.01); G06T 7/74 (2017.01); A61B 90/50 (2016.02); A61B 2034/104 (2016.02); A61B 2034/107 (2016.02); A61B 2034/2048 (2016.02); A61B 2034/2051 (2016.02); A61B 2034/2055 (2016.02); A61B 2090/364 (2016.02); A61B 2090/373 (2016.02); A61B 2090/3929 (2016.02); A61B 2090/3937 (2016.02); A61B 2090/3983 (2016.02); G06T 2207/10021 (2013.01); G06T 2207/10048 (2013.01); G06T 2207/10076 (2013.01); G06T 2207/10081 (2013.01); G06T 2207/10088 (2013.01); G06T 2207/10104 (2013.01); G06T 2207/10136 (2013.01); G06T 2207/30016 (2013.01); G06T 2207/30096 (2013.01); G06T 2207/30101 (2013.01); G06T 2207/30204 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method of detecting registration error during a medical procedure by way of a registration error detection system, the registration error detection system comprising a surgical navigation system and a computer control system, the computer control system comprising a processor, the navigation system comprising a tracking system, the tracking system comprising an imaging system and a tracking device, the imaging system comprising an actual camera having a virtual camera aligned therewith, and the registration error detection system operable with a display system, the display system comprising a display, the method comprising:
acquiring, in real-time, using the actual camera, a continuous intra-operatively acquired image stream of intraoperative images of a surgical field having a medical instrument as a real object, the at least one real object previously registered with at least one corresponding virtual object, the actual camera and the at least one real object previously registered, via the tracking system, to a common coordinate frame relative to a tracked reference marker, and the tracked reference marker disposed in a fixed position relative to a patient anatomy;
displaying the intraoperative image on the display;
acquiring a pre-operative image as a virtual image from an imaging device having at least one virtual object in the common coordinate frame;
overlaying, in real-time during the procedure, the virtual image in relation to the intraoperative image on the display;
detecting any misalignment between any one of the at least one real object of the intraoperative image and the corresponding at least one virtual object of the virtual image, wherein a presence of misalignment is indicative of registration error between the at least one virtual object and the corresponding at least one real object, and wherein a coordinate frame of the surgical navigation system is spatially registered with a coordinate frame of patient imaging data through respective alignment of corresponding pairs of at least one virtual point and at least one actual point;
automatically correcting the misalignment based on a difference between at least one local tissue characteristic and at least one virtual instrument representation at a location being focused and a live video stream of the surgical field, automatically correcting comprising: nonlinearly manipulating a rendering of a patient image and the virtual image to align with the intraoperative imaging; and nonlinearly manipulating comprising using optical flow to generate an alignment with the at least one real object, thereby providing an immediate local registration correction; and
globally applying the local registration correction to at least one of the imaging system, the tracking system, and the display system, thereby providing dynamic validation and re-registration, and thereby eliminating any need to reposition a patient.