US 12,223,472 B2
XR-based supervision assisting device, method and program
Jae Heon Jung, Seoul (KR); and Yu Mi Lee, Incheon (KR)
Assigned to Jae Heon Jung, Seoul (KR); Yu Mi Lee, Incheon (KR); and SLZ Inc., Siheung-si (KR)
Filed by Jae Heon Jung, Seoul (KR); Yu Mi Lee, Incheon (KR); and SLZ Inc., Incheon (KR)
Filed on Jun. 17, 2022, as Appl. No. 17/843,479.
Claims priority of application No. 10-2021-0080394 (KR), filed on Jun. 21, 2021.
Prior Publication US 2022/0405718 A1, Dec. 22, 2022
Int. Cl. G06T 19/00 (2011.01); G06Q 10/20 (2023.01); G06T 7/70 (2017.01); G06T 19/20 (2011.01); G06V 10/74 (2022.01); G06V 20/20 (2022.01)
CPC G06Q 10/20 (2013.01) [G06T 7/70 (2017.01); G06T 19/006 (2013.01); G06T 19/20 (2013.01); G06V 10/761 (2022.01); G06V 20/20 (2022.01); G06T 2207/30204 (2013.01); G06T 2210/04 (2013.01); G06T 2219/2021 (2013.01)] 18 Claims
OG exemplary drawing
 
1. An extended reality (XR)-based wearable supervision assisting device for assisting supervision of a construction site, the XR-based wearable supervision assisting device comprising:
a camera configured to photograph an image, wherein the camera comprises a gaze tracking camera configured to track a supervisor's gaze based on eye movements of the supervisor;
a transparent display unit;
a communication unit acquiring 3D-based building information modelling (BIM) design data for at least one real design object actually installed in the construction site; and
a processor performing operations of:
grasping the supervisor's gaze through the gaze tracking camera;
determining a real design object, which the supervisor is watching during a predetermined period of time or more, as a recognition target;
acquiring the BIM design data for the determined real design object through the communication unit;
rendering a virtual XR design object for the determined real design object on the basis of the acquired BIM design data;
displaying the rendered XR design object at a position corresponding to the determined real design object on the transparent display unit;
comparing the BIM design data of the XR design object with the photograph image, and checking an error between the XR design object and the determined real design object; and
informing the supervisor of that the checked error deviates from a predetermined allowable error range,
wherein the processor analyzes the BIM design data and derives a supervision progress direction from a starting position of the supervision,
wherein the processor determines a completion of correction of the error in the photographed image when correction of the error of the determined real design object that the supervisor is watching through the transparent display unit has been completed, and
wherein when the completion of correction of the error is determined, the processor renders and displays an XR content guiding a next supervision position on the basis of the supervision progress direction and a current position of the supervision assisting device.