US 12,008,750 B2
Generating annotation data of tissue images
Fei Zuo, Eindhoven (NL); Anke Pierik, Eindhoven (NL); Reinhold Wimberger-Friedl, Waalre (DE); Koen De Laat, Udenhout (NL); and Marc Van Driel, Nijmegen (NL)
Assigned to KONINKLIJKE PHILIPS N.V., Eindhoven (NL)
Appl. No. 17/267,846
Filed by KONINKLIJKE PHILIPS N.V., Eindhoven (NL)
PCT Filed Aug. 14, 2019, PCT No. PCT/EP2019/071769
§ 371(c)(1), (2) Date Feb. 11, 2021,
PCT Pub. No. WO2020/035514, PCT Pub. Date Feb. 20, 2020.
Claims priority of application No. 18189097 (EP), filed on Aug. 15, 2018.
Prior Publication US 2021/0312620 A1, Oct. 7, 2021
Int. Cl. G06T 7/00 (2017.01); G01N 1/30 (2006.01); G01N 1/34 (2006.01); G06T 7/11 (2017.01); G06T 11/00 (2006.01)
CPC G06T 7/0012 (2013.01) [G01N 1/30 (2013.01); G01N 1/34 (2013.01); G06T 7/11 (2017.01); G06T 11/00 (2013.01); G06T 2207/10056 (2013.01); G06T 2207/20072 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/30024 (2013.01); G06T 2207/30168 (2013.01)] 19 Claims
OG exemplary drawing
 
1. An apparatus for automatically generating annotation data of tissue images comprising:
a processor
a memory storing instructions, wherein the instructions when executed by the processor cause the processor to:
receive first image data comprising a representation of a tissue sample stained with a first stain for indicating tissue structure, and
receive second image data comprising a representation of the tissue sample stained with a second stain for revealing the presence of a biomarker in the tissue sample; and
detect a plurality of candidate objects in the first image data; and obtain a corresponding plurality of detection probabilities of the detected candidate objects,
generate stain confidence data of the second image data representing a likelihood that a corresponding portion of the tissue sample comprises the biomarker targeted by the second stain, and to generate image annotation data comprising a representation of the plurality of candidate objects, wherein the individual candidate objects of the plurality of candidate objects are annotated to indicate biomarker presence in corresponding portions of the second image data if an individual candidate object reaches a combined confidence based on the corresponding plurality of detection probabilities of the detected candidate objects, the corresponding stain confidence data, and a corresponding registration quality threshold of a registration between the first and second image data of the candidate objects;
segment the plurality of candidate objects in the first image data,
transfer the segmentation of the plurality of candidate objects in the first image data to the second image data based on the registration between the first and second image data, and
sum the corresponding stain confidence data in the location of the plurality of candidate objects in the second image data, wherein the summed stain confidence data is useable with the plurality of detection probabilities of the detected candidate objects to determine a likelihood score value for candidate objects to be positive for the biomarker targeted by the second stain; and
generate a registration quality map between the first image data and the second image data, wherein the processor is configured to only annotate candidate objects of the plurality of candidate objects to indicate biomarker presence when the registration quality map indicates that the registration between corresponding locations of the candidate object in the first and second image data surpasses a registration quality threshold.