US 12,469,315 B2
Systems, methods, and apparatuses for implementing transferable visual words by exploiting the semantics of anatomical patterns for self-supervised learning
Fatemeh Haghighi, Tempe, AZ (US); Mohammad Reza Hosseinzadeh Taher, Tempe, AZ (US); Zongwei Zhou, Tempe, AZ (US); and Jianming Liang, Scottsdale, AZ (US)
Assigned to Arizona Board of Regents on behalf of Arizona State University, Scottsdale, AZ (US)
Filed by Arizona Board of Regents on behalf of Arizona State University, Scottsdale, AZ (US)
Filed on Feb. 19, 2022, as Appl. No. 17/676,134.
Application 17/676,134 is a continuation in part of application No. 17/246,032, filed on Apr. 30, 2021.
Claims priority of provisional application 63/151,567, filed on Feb. 19, 2021.
Claims priority of provisional application 63/110,265, filed on Nov. 5, 2020.
Claims priority of provisional application 63/018,335, filed on Apr. 30, 2020.
Prior Publication US 2022/0309811 A1, Sep. 29, 2022
Int. Cl. G06V 20/70 (2022.01); G06V 10/26 (2022.01); G06V 10/74 (2022.01); G06V 10/764 (2022.01); G06V 10/774 (2022.01); G06V 10/82 (2022.01)
CPC G06V 20/70 (2022.01) [G06V 10/26 (2022.01); G06V 10/761 (2022.01); G06V 10/764 (2022.01); G06V 10/774 (2022.01); G06V 10/82 (2022.01); G06V 2201/03 (2022.01)] 8 Claims
OG exemplary drawing
 
1. A system comprising:
a memory to store instructions;
a processor to execute the instructions stored in the memory;
a receive interface to receive a plurality of unlabeled medical images obtained from a plurality of human patients;
wherein the system is specially configured to perform self-supervised learning for an artificial intelligence (AI) model in the absence of manually labeling the plurality of unlabeled medical images, by executing instructions via the processor for:
selecting a subset of the plurality of unlabeled medical images, corresponding to similar patients, based on deep latent features therein;
extracting two-dimensional (2D) patches or three-dimensional (3D) cubes each representing an appearance of an anatomical pattern reoccurring at fixed coordinates across each of the selected subset of the plurality of unlabeled medical images, and assigning one of a plurality of labels to each of the 2D patches or 3D cubes based on their fixed coordinates;
transforming each of the 2D patches or 3D cubes to generate transformed 2D patches or transformed 3D cubes, respectively
by perturbing each of the plurality of anatomical patterns resulting in a plurality of perturbed anatomical patterns;
performing, via an encoder-decoder network having skip connections in between and a classification head at an end of the encoder, a self-classification operation on the plurality of perturbed anatomical patterns by formulating a multi-class classification task that discriminates among the plurality of perturbed anatomical patterns based on their respective label to learn anatomical pattern semantics from the plurality of perturbed anatomical patterns and generate a corresponding plurality of latent representations;
performing a self-restoration operation that receives the plurality of latent representation and recovers the corresponding plurality of anatomical patterns from the plurality of perturbed anatomical patterns to learn image representation from multiple perspectives and encode anatomical diversity in the plurality of anatomical patterns.