US 11,727,674 B2
Systems and methods for generating histology image training datasets for machine learning models
Aïcha Bentaieb, Mountain View, CA (US); Martin Christian Stumpe, Belmont, CA (US); and Aly Azeem Khan, Chicago, IL (US)
Assigned to TEMPUS LABS, INC., Chicago, IL (US)
Filed by TEMPUS LABS, INC., Chicago, IL (US)
Filed on Dec. 13, 2021, as Appl. No. 17/549,040.
Claims priority of provisional application 63/199,185, filed on Dec. 11, 2020.
Prior Publication US 2022/0189150 A1, Jun. 16, 2022
Int. Cl. G16C 20/70 (2019.01); G06V 20/69 (2022.01); G06V 10/774 (2022.01)
CPC G06V 10/7747 (2022.01) [G06V 20/698 (2022.01); G16C 20/70 (2019.02)] 30 Claims
 
1. A method for using a machine learning model to analyze at least one hematoxylin and eosin (H&E) slide image, the method comprising:
a. receiving, at one or more processors, the H&E slide image and providing the H&E slide image to a machine learning model;
b. predicting, at the one or more processors, locations of molecules in the H&E slide image using the machine learning model where the machine learning model is trained using a training data set comprising a plurality of unmarked H&E images and a plurality of marked H&E images, each marked H&E image being associated with one unmarked H&E image and each marked H&E image including a location of one or more molecules determined by analyzing a multiplex IHC image having at least two IHC stains, wherein each IHC stain has a unique color and a unique target molecule and wherein analyzing the multiplex IHC image includes determining an IHC stain that contributes to any two or more overlapping or adjacent IHC stains and comparing each IHC stain in the multiplex IHC image to a threshold;
c. analyzing the number of predicted molecules and locations of the predicted molecules predicted by the machine learning model; and
d. assigning an immunotherapy response class to the H&E slide image, based on the number of predicted molecules and/or locations of the predicted molecules.