| CPC G06V 20/50 (2022.01) [G06T 7/73 (2017.01); G06T 17/20 (2013.01); G06T 19/006 (2013.01); G06V 10/70 (2022.01); G08B 17/125 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/30196 (2013.01); G06T 2207/30244 (2013.01); G06T 2210/56 (2013.01)] | 18 Claims |
|
1. A system for Mixed Reality (MR) and Artificial Intelligence (AI)-enhanced fire investigation, comprising:
a sensor device; and
an MR device in wireless communication with the sensor device, the MR device, comprising:
a processing device;
a wireless communication device in communication with the processing device and in selective communication with the sensor device;
a head-mounted see-through display in communication with the processing device;
a Time of Flight (ToF) sensor in communication with the processing device;
a camera in communication with the processing device;
one or more Inertial Measurement Unit (IMU) devices in communication with the processing device;
a battery in communication with the processing device; and
a memory device in communication with the processing device, the memory device storing MR instructions that when executed by the processing device, result in:
acquiring, by the ToF sensor and at a first time and from a first location in an environment in which a wearer of the head-mounted see-through display is located, data descriptive of first distances from the MR device to a first plurality of surface points in the environment, wherein the first plurality of surface points are within a first field of view of the camera;
computing, by the processing device and utilizing the first distances and the first location, a first portion of a 3-D point cloud descriptive of locations of the first plurality of surface points in the environment;
tracking, after the acquiring of the first distances and by the one or more IMU devices, a first movement of the wearer from the first location in the environment to a second location in the environment;
acquiring, by the ToF sensor and at a second time and from the second location in the environment, data descriptive of second distances from the MR device to a second plurality of surface points in the environment, wherein the second plurality of surface points are within a second field of view of the camera;
computing, by the processing device and utilizing the second distances and the second location, a second portion of the 3-D point cloud descriptive of locations of the second plurality of surface points in the environment;
generating, by the processing device and utilizing the first and second portions of the 3-D point cloud, a 3-D wire mesh model descriptive of the environment;
receiving, by the wireless communication device and from the sensor device, (i) data descriptive of the environment that has been captured by the sensor device and (ii) positioning information descriptive of a location and orientation of the sensor device;
identifying, by the processing device and based on the positioning information descriptive of a location and orientation of the sensor device, a portion of the 3-D wire mesh model that corresponds to the data descriptive of the environment that has been captured by the sensor device; and
assigning, by the processing device and to the corresponding portion of the 3-D wire mesh model, an attribute representative of the data descriptive of the environment that has been captured by the sensor device.
|