US 11,984,217 B2
Method and apparatus for processing histological image captured by medical imaging device
Kenneth Mark Sullivan, San Diego, CA (US); and Jin Man Kang, San Diego, CA (US)
Assigned to KOH YOUNG TECHNOLOGY INC., Seoul (KR)
Appl. No. 16/487,636
Filed by KOH YOUNG TECHNOLOGY INC., Seoul (KR)
PCT Filed Feb. 21, 2018, PCT No. PCT/KR2018/002118
§ 371(c)(1), (2) Date Aug. 21, 2019,
PCT Pub. No. WO2018/155898, PCT Pub. Date Aug. 30, 2018.
Claims priority of provisional application 62/563,751, filed on Sep. 27, 2017.
Claims priority of provisional application 62/461,490, filed on Feb. 21, 2017.
Prior Publication US 2020/0035350 A1, Jan. 30, 2020
Int. Cl. G16H 30/40 (2018.01); G06N 3/08 (2023.01); G06N 20/00 (2019.01); G06T 11/00 (2006.01); G16H 50/20 (2018.01); G16H 50/50 (2018.01); G16H 30/20 (2018.01); G16H 70/00 (2018.01)
CPC G16H 30/40 (2018.01) [G06N 3/08 (2013.01); G06N 20/00 (2019.01); G06T 11/001 (2013.01); G16H 50/20 (2018.01); G16H 50/50 (2018.01); G06T 2210/41 (2013.01); G16H 30/20 (2018.01); G16H 70/00 (2018.01)] 7 Claims
OG exemplary drawing
 
1. A method, performed by an image processing device comprising a processor, a storage, and a communication unit, for processing a histological image captured by a medical imaging device, the method comprising:
receiving the histological image including at least one type of tissue;
determining, by a first autoencoder of the processor, a candidate type of tissue in the histological image;
identifying, by the first autoencoder of the processor, a target region corresponding to the candidate type of tissue in the histological image;
identifying at least one target histological image corresponding to the target region in the histological image based on a predictive model associating one or more sample histological images with one or more sample target histological images;
applying one or more display characteristics associated with the identified at least one target histological image to the histological image; and
generating a modified image of the histological image including the applied one or more display characteristics,
wherein a first unsupervised learning model is generated by training the first autoencoder of the processor based on a first set of sample histological images, and a second unsupervised learning model is generated by training a second autoencoder of the processor based on a second set of sample target histological images,
wherein the predictive model is generated by associating the second unsupervised learning model with the first unsupervised learning model based on a set of matching images,
wherein the set of matching images include M sample histological images in the first set of sample histological images and N sample target histological images in the second set of sample target histological images,
wherein one or more anatomical locations of the M sample histological images in the first set of sample histological images are aligned to match one or more anatomical locations of the N sample target histological images in the second set of sample target histological images, M and N being positive integers,
wherein the predictive model comprises data regarding one or more features indicative of at least one display characteristic and is trained by associating one or more features from the N sample target histological images with one or more features from the M sample histological images,
wherein the predictive model comprises a first set of patches in each of the one or more sample histological images and a second set of patches in each of the one or more sample target histological images,
wherein the first set of patches is associated with the second set of patches in the predictive model,
wherein applying the one or more display characteristics comprises modifying a plurality of patches in the received histological image based on the second set of patches in the identified at least one target histological image,
wherein the first unsupervised learning model is trained based on one or more features associated with the target region in the histological image, and
wherein the identified at least one target histological image includes a staining image.