US 12,080,184 B2
Mixed-reality endoscope and surgical tools with haptic feedback for integrated virtual-reality visual and haptic surgical simulation
Samuel Barber, Tucson, AZ (US); Saurabh Jain, Tucson, AZ (US); Young-Jun Son, Tucson, AZ (US); and Eugene Chang, Tucson, AZ (US)
Assigned to Arizona Board of Regents on Behalf of the University of Arizona, Tucson, AZ (US)
Appl. No. 17/048,962
Filed by ARIZONA BOARD OF REGENTS ON BEHALF OF THE UNIVERSITY OF ARIZONA, Tucson, AZ (US)
PCT Filed Apr. 18, 2019, PCT No. PCT/US2019/028127
§ 371(c)(1), (2) Date Oct. 19, 2020,
PCT Pub. No. WO2019/204607, PCT Pub. Date Oct. 24, 2019.
Claims priority of provisional application 62/659,672, filed on Apr. 18, 2018.
Claims priority of provisional application 62/659,680, filed on Apr. 18, 2018.
Claims priority of provisional application 62/659,685, filed on Apr. 18, 2018.
Prior Publication US 2021/0241656 A1, Aug. 5, 2021
Int. Cl. G09B 23/28 (2006.01); A61B 17/24 (2006.01); A61B 34/10 (2016.01); A61B 34/20 (2016.01); A61B 90/00 (2016.01); B33Y 80/00 (2015.01); G06F 3/01 (2006.01); G06F 3/0346 (2013.01); G06T 7/11 (2017.01); G09B 23/30 (2006.01); G16H 20/40 (2018.01); G16H 40/63 (2018.01); G16H 50/50 (2018.01); A61B 17/00 (2006.01); A61B 17/29 (2006.01); A61B 17/32 (2006.01); A61B 17/3201 (2006.01); A61B 18/00 (2006.01); A61B 18/14 (2006.01); A61B 90/50 (2016.01)
CPC G09B 23/285 (2013.01) [A61B 17/24 (2013.01); A61B 34/10 (2016.02); A61B 34/20 (2016.02); A61B 90/36 (2016.02); B33Y 80/00 (2014.12); G06F 3/011 (2013.01); G06F 3/012 (2013.01); G06F 3/016 (2013.01); G06F 3/0346 (2013.01); G06T 7/11 (2017.01); G09B 23/30 (2013.01); G16H 20/40 (2018.01); G16H 40/63 (2018.01); G16H 50/50 (2018.01); A61B 2017/00119 (2013.01); A61B 2017/00216 (2013.01); A61B 2017/00707 (2013.01); A61B 2017/00716 (2013.01); A61B 17/29 (2013.01); A61B 17/320016 (2013.01); A61B 17/3201 (2013.01); A61B 2018/00297 (2013.01); A61B 18/14 (2013.01); A61B 2034/101 (2016.02); A61B 2034/105 (2016.02); A61B 2034/107 (2016.02); A61B 2090/365 (2016.02); A61B 2090/372 (2016.02); A61B 2090/502 (2016.02); G06T 2207/10068 (2013.01); G06T 2207/10072 (2013.01); G06T 2207/30004 (2013.01)] 13 Claims
OG exemplary drawing
 
1. A method of preparing a physical model of a head and endoscope for surgical simulation comprising:
importing into a workstation a radiological tomographic image stack of a head;
segmenting the radiological tomographic image stack into hard tissue, soft tissue, and mucosal voxel models based at least in part on voxel intensity;
growing hard tissue, mucosal, and soft tissue regions in the hard tissue, soft tissue, and mucosal voxel models;
converting the hard tissue, soft tissue, and mucosal models into a hard tissue mesh model, a soft tissue mesh model, and a mucosal mesh model;
repairing the mesh models;
exporting the mesh models from the workstation and using at least a 3D printer and the hard tissue mesh model to generate a physical head model by:
printing a hard plastic physical hard tissue model from the hard tissue mesh model,
preparing a resilient polymer physical mucosal tissue model from the mucosal mesh model;
mounting the physical mucosal tissue model to the physical hard tissue model;
preparing a resilient polymer physical soft tissue model from the soft tissue mesh model; and
mounting the physical soft tissue model to the physical hard tissue model to form the physical head model;
loading the mesh models into a display system adapted to render images of surfaces of the mesh models as viewed from an endoscope;
mounting a tracker to the physical head model; and
preparing an endoscope device with a tracker.