US 12,456,190 B2
Automated assessment of endoscopic disease
Fillippo Arcadu, Basel (CH); Benjamin Gutierrez-Becker, Basel (CH); Andreas Thalhammer, Basel (CH); Marco Prunotto, South San Francisco, CA (US); and Young Suk Oh, South San Francisco, CA (US)
Assigned to HOFFMANN-LA ROCHE INC., Little Falls, NJ (US); and GENENTECH, INC., South San Francisco, CA (US)
Appl. No. 17/797,293
Filed by Genentech, Inc., South San Francisco, CA (US); and Hoffmann-La Roche Inc., Little Falls, NJ (US)
PCT Filed Jan. 29, 2021, PCT No. PCT/EP2021/052170
§ 371(c)(1), (2) Date Aug. 3, 2022,
PCT Pub. No. WO2021/156152, PCT Pub. Date Aug. 12, 2021.
Claims priority of application No. 20155469 (EP), filed on Feb. 4, 2020.
Prior Publication US 2023/0047100 A1, Feb. 16, 2023
Int. Cl. G06T 7/00 (2017.01); A61B 1/00 (2006.01); A61B 1/31 (2006.01)
CPC G06T 7/0012 (2013.01) [A61B 1/000096 (2022.02); A61B 1/31 (2013.01); G06T 2207/10016 (2013.01); G06T 2207/10068 (2013.01); G06T 2207/20076 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30032 (2013.01); G06T 2207/30096 (2013.01); G06T 2207/30168 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method of analyzing a colonoscopy video or a portion thereof, the method comprising:
using a first deep neural network classifier to classify image data from the colonoscopy video or portion thereof into one of at least a first severity class and a second severity class, the first severity class being associated with more severe endoscopic lesions than the second severity class, wherein the first deep neural network classifier has been trained at least in part in a weakly supervised manner using training image data from a plurality of training colonoscopy videos, wherein the training image data comprises multiple sets of consecutive frames from the plurality of training colonoscopy videos, wherein each of the frames in a set has a severity class label, and wherein all of the frames in a given set have the same severity class label; and
using a second deep neural network classifier to classify the image data from the colonoscopy video or portion thereof into one of at least a first quality class and a second quality class, wherein the first quality class is associated with better quality images than the second quality class, wherein image data in the first quality class is provided to the first deep neural network classifier, or wherein the second deep neural network classifier has been trained at least in part in a weakly supervised manner using training image data from a plurality of training colonoscopy videos, wherein the training image data comprises multiple sets of consecutive frames from the plurality of training colonoscopy videos, wherein each frame in a set are associated with a quality class label that is the same for all frames in the set, and wherein each set of consecutive frames in the training image data has been assigned a quality class label by visual inspection of the segment of video comprising the respective set of consecutive frames.