US 12,032,658 B2
Method and system for improving cancer detection using deep learning
Atilla Kiraly, Mountain View, CA (US); Shravya Shetty, Mountain View, CA (US); Sujeeth Bharadwaj, Mountain View, CA (US); Diego Ardila, Mountain View, CA (US); and Bokyung Choi, Mountain View, CA (US)
Assigned to Google LLC, Mountain View, CA (US)
Appl. No. 17/055,313
Filed by GOOGLE LLC, Monntain View, CA (US)
PCT Filed Nov. 20, 2018, PCT No. PCT/US2018/061956
§ 371(c)(1), (2) Date Nov. 13, 2020,
PCT Pub. No. WO2019/245597, PCT Pub. Date Dec. 26, 2019.
Claims priority of provisional application 62/726,178, filed on Aug. 31, 2018.
Claims priority of provisional application 62/686,541, filed on Jun. 18, 2018.
Prior Publication US 2021/0225511 A1, Jul. 22, 2021
Int. Cl. G06T 7/11 (2017.01); G06F 18/213 (2023.01); G06F 18/2415 (2023.01); G06F 18/25 (2023.01); G06N 3/04 (2023.01); G06N 3/08 (2023.01); G06N 20/20 (2019.01); G06T 7/00 (2017.01); G06T 7/73 (2017.01); G16H 50/20 (2018.01)
CPC G06F 18/254 (2023.01) [G06F 18/213 (2023.01); G06F 18/2415 (2023.01); G06N 3/04 (2013.01); G06N 3/08 (2013.01); G06N 20/20 (2019.01); G06T 7/0012 (2013.01); G06T 7/11 (2017.01); G06T 7/73 (2017.01); G16H 50/20 (2018.01); G06T 2207/10081 (2013.01); G06T 2207/20076 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/20132 (2013.01); G06T 2207/30061 (2013.01); G06T 2207/30096 (2013.01); G06V 2201/031 (2022.01)] 33 Claims
OG exemplary drawing
 
1. A method for improving lung cancer screening and detection from a computed tomography data set obtained for a patient, comprising the steps of
a) supplying the data set to a global predictive model comprising a three-dimensional deep convolutional neural network trained to predict at least the probability of the presence of cancer in lung tissue in the data set on a global basis, wherein the global predictive model includes an intermediate convolutional layer associated with a feature map;
b) supplying the data set to a two-stage prediction model, the two-stage prediction model comprising
1) a first stage detection model configured to detect the location of and generate respective cropped 3D volumes for one or more three-dimensional cancer candidates within the data set, and
2) a second stage probability model operating on the one or more cropped 3D volumes for each of the one or more three-dimensional cancer candidates detected by the first stage detection model and assigning a cancer probability p to each of the three-dimensional cancer candidates, wherein the second stage probability model comprises an intermediate convolutional layer associated with a further feature map; and
c) generating data representing (1) an overall prediction of the probability of cancer in the data set using both the prediction of the global predictive model and the cancer probabilities p assigned by the second stage probability model to each of the three-dimensional cancer candidates, wherein the overall prediction of the probability of cancer in the data set is obtained by either (i) appending the further feature map from the second stage model to the feature map of the intermediate convolutional layer of the global predictive model and generating the prediction from the global model, or (ii) appending the feature map from the global predictive model to the further feature map of the intermediate convolutional layer of the second stage model, and generating the prediction from the output of the second stage model, and (2) the location of cancer in the data set, wherein the location of cancer is determined by either the global predictive model or the two-stage prediction model.