US 12,223,642 B2
Deep learning model to predict data from an image
Shreya Sharma, Uttarakhand (IN); Abhishek Vahadane, Maharashtra (IN); Srikanth Ragothaman, Tamilnadu (IN); and Shantanu Majumdar, West Bengal (IN)
Assigned to Rakuten Group, Inc., Tokyo (JP)
Filed by Rakuten Group, Inc., Tokyo (JP)
Filed on May 3, 2021, as Appl. No. 17/306,170.
Prior Publication US 2022/0351366 A1, Nov. 3, 2022
Int. Cl. G06T 3/40 (2024.01); G06N 3/08 (2023.01); G06T 7/00 (2017.01); G06T 7/73 (2017.01)
CPC G06T 7/0012 (2013.01) [G06N 3/08 (2013.01); G06T 3/40 (2013.01); G06T 7/73 (2017.01); G06T 2207/10056 (2013.01); G06T 2207/20021 (2013.01); G06T 2207/20081 (2013.01)] 11 Claims
OG exemplary drawing
 
1. A method of predicting data from an image, executable by a processor, comprising:
dividing a hematoxylin and eosin (H&E) image into one or more patch images;
compressing, using a first model, spatial features corresponding to the one or more patch images; and
predicting, using a second model, output molecular data corresponding to the compressed spatial features; and
adjusting parameters of the first model based on minimizing a first loss function corresponding to the compressed spatial features and adjusting parameters of the second model based on minimizing a second loss function corresponding to the compressed spatial features,
wherein the first loss function comprises a regression loss function corresponding to compressed feature expression images output by the first model based on the one or more patch images, and
wherein the second loss function comprises a feature compression loss function corresponding to second output data, output by the second model, indicated by the H&E image based on the compressed feature expression images.