US 12,327,411 B2
Converting spatial information to haptic and auditory feedback
Kavan Hazeli, Tucson, AZ (US); Bahram Jalali, Los Angeles, CA (US); and Janet Roveda, Tucson, AZ (US)
Assigned to Arizona Board of Regents on Behalf of the University of Arizona, Tucson, AZ (US)
Filed by Arizona Board of Regents on Behalf of the University of Arizona, Tucson, AZ (US)
Filed on Aug. 22, 2023, as Appl. No. 18/236,842.
Claims priority of provisional application 63/399,901, filed on Aug. 22, 2022.
Prior Publication US 2024/0062548 A1, Feb. 22, 2024
Int. Cl. G06V 20/50 (2022.01); G01S 17/08 (2006.01); G01S 17/86 (2020.01); G06V 10/82 (2022.01); G08B 7/00 (2006.01)
CPC G06V 20/50 (2022.01) [G01S 17/08 (2013.01); G01S 17/86 (2020.01); G06V 10/82 (2022.01); G08B 7/00 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system for sensing objects in an environment and outputting haptic and auditory feedback indicative of the distance and direction of those objects, the system comprising:
a wearable image capture system comprising:
one or more light detection and ranging (LiDAR) scanners that sense the depths of objects in the environment and the angles of those depths relative to an angle of orientation of the wearable image capture system; and
one or more cameras that capture image data of the environment;
a neural network trained to detect objects in the image data and generate an object description of each detected object;
a haptic feedback system comprising a haptic feedback device associated with each of a plurality of directions, each direction associated with an angle range relative to the angle of orientation of the wearable image capture system;
a hardware computer processor that:
identifies the direction of the object by comparing the angle of the object to the angle ranges associated with each of the plurality of directions; and
outputs haptic feedback via the haptic feedback device associated with the direction of the object; and
an auditory feedback device that outputs the object description of each detected object.