CPC G16H 40/20 (2018.01) [G06N 3/02 (2013.01); G06N 3/045 (2023.01); G06N 3/08 (2013.01); G06T 7/0016 (2013.01); G06V 10/774 (2022.01); G06V 10/7715 (2022.01); G06V 10/82 (2022.01); G16H 30/40 (2018.01); G16H 50/20 (2018.01); G16H 50/70 (2018.01); G06N 20/00 (2019.01); G06T 2207/10088 (2013.01); G06T 2207/10104 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30016 (2013.01); G06T 2207/30104 (2013.01); G06V 2201/03 (2022.01)] | 20 Claims |
1. A method performed by one or more computers, the method comprising:
receiving multi-modal data characterizing a patient, wherein the multi-modal data comprises a respective feature representation for each of a plurality of modalities;
processing the multi-modal data characterizing the patient using a machine learning model, in accordance with values of a set of machine learning model parameters, to generate a patient classification that classifies the patient as being included in a patient category from a set of patient categories, comprising:
processing the multi-modal data characterizing the patient using an encoder neural network of the machine learning model to generate an embedding of the multi-modal data in a latent space; and
generating the patient classification based on the embedding of the multi-modal data in the latent space;
wherein the encoder neural network has been trained on a plurality of training examples by a machine learning training technique, the training comprising, for each training example:
processing training multi-modal data included in the training example using the encoder neural network to generate an embedding of the training multi-modal data;
processing the embedding of the training multi-modal data using a decoder neural network to generate a reconstruction of the training multi-modal data; and
updating current values of a set of encoder neural network parameters using gradients of an objective function that measures an error in the reconstruction of the training multi-modal data;
determining an uncertainty measure that characterizes an uncertainty of the patient classification generated by the machine learning model; and
generating a clinical recommendation for medical treatment of the patient based on: (i) the patient classification, and (ii) the uncertainty measure that characterizes the uncertainty of the patient classification.
|