US 12,412,407 B2
Systems and methods for classifying and annotating images taken during a medical procedure
Brian Fouts, Morgan Hill, CA (US); Cole Kincaid Hunter, Santa Clara, CA (US); and Harmandip Singh Sodhi, San Jose, CA (US)
Assigned to Stryker Corporation, Portage, MI (US)
Filed by Stryker Corporation, Kalamazoo, MI (US)
Filed on Dec. 29, 2021, as Appl. No. 17/565,389.
Claims priority of provisional application 63/132,445, filed on Dec. 30, 2020.
Prior Publication US 2022/0207896 A1, Jun. 30, 2022
Int. Cl. G06V 20/70 (2022.01); G06F 18/214 (2023.01); G06N 20/00 (2019.01); G06T 7/00 (2017.01); G06V 10/764 (2022.01); G06V 20/40 (2022.01); G16H 30/40 (2018.01)
CPC G06V 20/70 (2022.01) [G06F 18/214 (2023.01); G06N 20/00 (2019.01); G06T 7/0012 (2013.01); G06V 10/764 (2022.01); G06V 20/41 (2022.01); G16H 30/40 (2018.01); G06T 2207/10016 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/30004 (2013.01); G06T 2207/30168 (2013.01); G06V 2201/03 (2022.01)] 24 Claims
OG exemplary drawing
 
1. A method for annotating one or more images generated during a surgical procedure to provide additional information to a viewer of the images, the method comprising:
receiving a selection of a pre-defined template for a report corresponding to the surgical procedure;
receiving video data captured from an imaging tool configured to image an internal portion of a patient;
extracting one or more image frames from the received video data;
applying one or more machine learning classifiers to the received video data to generate one or more classification metrics based on the received video data, wherein the one or more machine learning classifiers are created using a supervised training process that comprises using one or more annotated images to train the machine learning classifier, wherein the one or more machine learning classifiers comprise a joint type machine learning classifier configured to generate one or more classification metrics associated with identifying a type of joint pictured in the received video data;
identifying one or more characteristics in the one or more image frames based on the generated one or more classification metrics;
determining that the one or more image frames correspond to one or more annotations that provide additional information to a viewer of the one or more image frames based on the identified one or more characteristics; and
inserting the one or more image frames into the pre-defined template in association with the one or more annotations.