US 11,915,408 B2
Methods of artificial intelligence-assisted infrastructure assessment using mixed reality systems
Enes Karaaslan, Orlando, FL (US); Fikret Necati Catbas, Orlando, FL (US); and Ulas Bagci, Orlando, FL (US)
Assigned to University of Central Florida Research Foundation, Inc., Orlando, FL (US)
Filed by University of Central Florida Research Foundation, Inc., Orlando, FL (US)
Filed on Dec. 28, 2022, as Appl. No. 18/147,256.
Application 18/147,256 is a division of application No. 17/247,389, filed on Dec. 9, 2020, granted, now 11,551,344.
Claims priority of provisional application 62/945,568, filed on Dec. 9, 2019.
Prior Publication US 2023/0214983 A1, Jul. 6, 2023
Int. Cl. G06T 7/00 (2017.01); G06F 3/01 (2006.01); G06T 7/11 (2017.01); G02B 27/01 (2006.01); G06V 10/94 (2022.01); G06V 10/774 (2022.01)
CPC G06T 7/0004 (2013.01) [G02B 27/0172 (2013.01); G06F 3/011 (2013.01); G06T 7/11 (2017.01); G06V 10/774 (2022.01); G06V 10/95 (2022.01); G02B 2027/014 (2013.01); G02B 2027/0138 (2013.01); G02B 2027/0141 (2013.01); G06T 2200/24 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/30132 (2013.01); G06T 2210/12 (2013.01)] 3 Claims
OG exemplary drawing
 
1. A hybrid system for contactless structure inspection, the hybrid system comprising:
a front-end module including a mixed reality headset in communication with a user-input actuator, the mixed reality headset configured to scan a surrounding area and capture images within the surrounding area to detect a defect in a structure disposed within the surrounding area, the user-input actuator configured to receive a selection from a user to investigate a portion of the surrounding area, such that the mixed reality headset captures an image of the portion of the surrounding area; and
a back-end module including a server in wireless communication with the mixed reality headset, the server including a deep learning module having a trained data set of defects in structures, the server configured to receive the captured image of the portion of the surrounding area from the mixed reality headset, and the deep learning module configured to compare the captured image of the portion of the surrounding area with the trained data set of defects in structure to determine a presence of a defect,
wherein the hybrid system is configured to implement an attention-guided technique, such that the mixed reality headset scans the surrounding area for defects to guide an investigation into one or more defects within the surrounding area, and
wherein, upon receiving the selection from the user, the mixed reality headset creates a bounding box around the portion of the surrounding area, such that only the mixed reality headset transmits only the bounding box to the server for investigation.