US 11,893,724 B2
Methods of artificial intelligence-assisted infrastructure assessment using mixed reality systems
Enes Karaaslan, Orlando, FL (US); Fikret Necati Catbas, Orlando, FL (US); and Ulas Bagci, Orlando, FL (US)
Assigned to University of Central Florida Research Foundation, Inc., Orlando, FL (US)
Filed by University of Central Florida Research Foundation, Inc., Orlando, FL (US)
Filed on Dec. 28, 2022, as Appl. No. 18/147,279.
Application 18/147,279 is a division of application No. 17/247,389, filed on Dec. 9, 2020, granted, now 11,551,344.
Claims priority of provisional application 62/945,568, filed on Dec. 9, 2019.
Prior Publication US 2023/0131469 A1, Apr. 27, 2023
Int. Cl. G06T 7/00 (2017.01); G06F 3/01 (2006.01); G06T 7/11 (2017.01); G02B 27/01 (2006.01); G06V 10/94 (2022.01); G06V 10/774 (2022.01)
CPC G06T 7/0004 (2013.01) [G02B 27/0172 (2013.01); G06F 3/011 (2013.01); G06T 7/11 (2017.01); G06V 10/774 (2022.01); G06V 10/95 (2022.01); G02B 2027/014 (2013.01); G02B 2027/0138 (2013.01); G02B 2027/0141 (2013.01); G06T 2200/24 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/30132 (2013.01); G06T 2210/12 (2013.01)] 6 Claims
OG exemplary drawing
 
1. A method of calculating a likelihood of a defect within a captured image of a structure using a hybrid system, the method comprising the steps of:
scanning an area surrounding a mixed reality headset to capture one or more images including possible defects;
suggesting, via the mixed reality headset, an area within the captured one or more images for an analysis;
receiving, via a user-input actuator in communication with the mixed reality headset, an input from a user to investigate the area;
capturing, via the mixed reality headset, an image of the area;
generating, via the mixed reality headset, a bounding box surrounding the image of the area, the bounding box including a plurality of boundaries;
projecting, via the mixed reality headset, an object onto the bounding box surrounding the image of the area;
transmitting, via the mixed reality headset, the bounding box surrounding the image of the area and the projected object to a server;
segmenting, via the server, the image within the bounding box by creating a plurality of prediction vectors within the image;
comparing, via the server, the segmented image with a plurality of images from a trained data set by comparing the plurality of prediction vectors and the projected object to the trained data set;
calculating, via the server, a likelihood of a defect within the segmented image based on the comparison with the trained data set; and
transmitting, via the server, the calculated likelihood of the defect to the mixed reality headset.