US 11,935,232 B2
Predicting disease progression from tissue images and tissue segmentation maps
Jason Yim, London (GB); Reena Kumari Chopra, Hounslow (GB); Terry Spitz, London (GB); Jim Huibrecht Winkens, London (GB); Annette Ada Nkechinyere Obika, London (GB); Trevor Back, Saffron Walden (GB); Joseph R. Ledsam, Tokyo (JP); Pearse A. Keane, London (GB); and Jeffrey De Fauw, London (GB)
Assigned to Google LLC, Mountain View, CA (US)
Appl. No. 17/257,999
Filed by Google LLC, Mountain View, CA (US)
PCT Filed Aug. 17, 2020, PCT No. PCT/US2020/046599
§ 371(c)(1), (2) Date Jan. 5, 2021,
PCT Pub. No. WO2021/041068, PCT Pub. Date Mar. 4, 2021.
Claims priority of provisional application 62/894,562, filed on Aug. 30, 2019.
Prior Publication US 2022/0301152 A1, Sep. 22, 2022
Int. Cl. A61B 5/00 (2006.01); G06T 7/00 (2017.01); G06T 7/11 (2017.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01)
CPC G06T 7/0012 (2013.01) [A61B 5/4842 (2013.01); A61B 5/7275 (2013.01); G06T 7/11 (2017.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06T 2200/04 (2013.01); G06T 2207/10012 (2013.01); G06T 2207/10101 (2013.01); G06T 2207/20021 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30041 (2013.01); G06V 2201/03 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to implement:
one or more first classification neural networks, wherein each first classification neural network is configured to:
receive, by an input layer of the first classification neural network, an image of eye tissue captured using an imaging modality; and
process the image to generate, by an output layer of the first classification neural network, a first progression score characterizing a likelihood that a state of a medical condition affecting the eye tissue will progress to a target state in a future interval of time;
one or more second classification neural networks, wherein each second classification neural network is configured to:
receive, by an input layer of the second classification neural network, a segmentation map of an image of eye tissue that segments the eye tissue in the image into a plurality of tissue types; and
process the segmentation map to generate, by an output layer of the second classification neural network, a second progression score characterizing a likelihood that a state of a medical condition affecting the eye tissue will progress to a target state in a future interval of time;
a subsystem configured to:
obtain: (i) an input image of eye tissue captured using an imaging modality, and (ii) a segmentation map of the eye tissue in the input image into a plurality of tissue types; and
generate, based on the input image and the segmentation map, a final progression score characterizing a likelihood that a state of a medical condition affecting the eye tissue will progress to a target state in a future interval of time, comprising:
providing the input image to each of the first classification neural networks to obtain a respective first progression score from each first classification neural network;
providing the segmentation map to each of the second classification neural networks to obtain a respective second progression score from each second classification neural network; and
generating the final progression score based on: (i) the first progression scores obtained based on the input image of eye tissue, and (ii) the second progression scores obtained based on the segmentation map.