US 11,925,418 B2
Methods for multi-modal bioimaging data integration and visualization
Noam Peled, Charlestown, MA (US); and Steven Stufflebeam, Charlestown, MA (US)
Assigned to The General Hospital Corporation, Boston, MA (US)
Appl. No. 17/781,858
Filed by The General Hospital Corporation, Boston, MA (US)
PCT Filed Dec. 2, 2020, PCT No. PCT/US2020/062900
§ 371(c)(1), (2) Date Jun. 2, 2022,
PCT Pub. No. WO2021/113370, PCT Pub. Date Jun. 10, 2021.
Claims priority of provisional application 62/942,585, filed on Dec. 2, 2019.
Prior Publication US 2023/0008541 A1, Jan. 12, 2023
Int. Cl. G09B 19/24 (2006.01); A61B 34/10 (2016.01); G01R 33/48 (2006.01); G03H 1/00 (2006.01); G06T 7/33 (2017.01); G06T 19/00 (2011.01); G09B 5/02 (2006.01); G09B 23/30 (2006.01); A61B 90/00 (2016.01)
CPC A61B 34/10 (2016.02) [G01R 33/4806 (2013.01); G03H 1/0005 (2013.01); G06T 7/344 (2017.01); G06T 19/006 (2013.01); G09B 5/02 (2013.01); G09B 19/24 (2013.01); A61B 2034/104 (2016.02); A61B 2034/105 (2016.02); A61B 2090/365 (2016.02); G03H 2001/0061 (2013.01); G06T 2207/30004 (2013.01); G09B 23/30 (2013.01)] 22 Claims
OG exemplary drawing
 
1. A system for simulating a medical procedure for a medical professional to carry out using a medical tool, the system comprising:
a computer system configured to:
(i) access anatomical imaging data acquired from a subject using a first imaging modality;
(ii) access physiological imaging data acquired from a subject using a second imaging modality;
(iii) combine the first imaging data and the second imaging data to generate an integrated model of the subject;
one of a virtual reality and augmented reality system configured to:
(i) register the integrated model of the subject with the subject and generate a visual representation of the subject using the integrated model of the subject for a medical professional to experience using the one of virtual reality and augmented reality system;
(ii) register, with the visual representation of the subject, a model of a medical tool corresponding to a medical tool used by the medical professional using the one of virtual reality and augmented reality system;
(iii) update the visual representation of the subject based on a virtual interaction of the medical tool used by the medical professional with the visual representation of the subject using the model of the subject and the model of the medical tool;
(iv) generate an updated visual representation of the subject after the virtual interaction of the medical tool used by the medical professional with the visual representation of the subject using the model of the subject and the model of the medical tool to illustrate the medical procedure and
(v) detecting facial points of the subject and registering the visual representation of the subject with the subject by co-registering the facial points of the subject and the anatomical imaging data.