US 11,914,369 B2
Multi-sensor environmental mapping
Ang Liu, Shenzhen (CN); Weiyu Mo, Shenzhen (CN); and Yonggen Ling, Shenzhen (CN)
Assigned to SZ DJI TECHNOLOGY CO., LTD., Shenzhen (CN)
Filed by SZ DJI TECHNOLOGY CO., LTD., Shenzhen (CN)
Filed on Jan. 25, 2021, as Appl. No. 17/157,905.
Application 17/157,905 is a continuation of application No. 16/574,848, filed on Sep. 18, 2019, granted, now 10,901,419.
Application 16/574,848 is a continuation of application No. 14/814,159, filed on Jul. 30, 2015, granted, now 10,429,839, issued on Oct. 1, 2019.
Application 14/814,159 is a continuation of application No. PCT/CN2014/086007, filed on Sep. 5, 2014.
Prior Publication US 2021/0247764 A1, Aug. 12, 2021
Int. Cl. G05D 1/00 (2006.01); G05D 1/10 (2006.01); G01C 21/00 (2006.01); B64C 39/02 (2023.01); G01S 19/42 (2010.01)
CPC G05D 1/0088 (2013.01) [B64C 39/024 (2013.01); G01C 21/3848 (2020.08); G05D 1/102 (2013.01); B64U 2201/10 (2023.01); G01S 19/42 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for controlling an unmanned aerial vehicle (UAV) in an environment, the method comprising:
receiving a first sensing signal from a vision sensor coupled to the UAV and a second sensing signal from a proximity sensor coupled to the UAV, the first sensing signal including first depth information of the environment and the second sensing signal including second depth information of the environment;
selecting the first sensing signal for generating a first portion of an environmental map and the second sensing signal for generating a second portion of the environmental map based on a suitable criterion, wherein the suitable criterion is associated with distinct characteristics of various portions of the environment or distinct capabilities of the vision sensor and the proximity sensor, the first sensing signal is more suitable than the second sensing signal for generating the first portion of the environmental map, the second sensing signal is more suitable than the first sensing signal for generating the second portion of the environmental map, and the first portion is different from the second portion;
generating a first set of depth images for the first portion of the environmental map based on the first sensing signal and a second set of depth images for the second portion of the environmental map based on the second sensing signal, the first set of depth images including a portion of the first depth information corresponding to the first portion of the environmental map and the second set of depth images including a portion of the second depth information corresponding to the second portion of the environmental map;
combining the first set of depth images and the second set of depth images to generate the environmental map; and
effecting the UAV to navigate in the environment using the environmental map.
 
11. A system for controlling an unmanned aerial vehicle (UAV) in an environment, the system comprising:
a plurality of sensors coupled to the UAV and including at least a vision sensor and a proximity sensor; and
one or more processors, individually or collectively, configured to:
receive a first sensing signal from the vision sensor and a second sensing signal from the proximity sensor, the first sensing signal including first depth information of the environment and the second sensing signal including second depth information of the environment;
select the first sensing signal for generating a first portion of an environmental map and the second sensing signal for generating a second portion of the environmental map based on a suitable criterion, wherein the suitable criterion is associated with distinct characteristics of various portions of the environment or distinct capabilities of the vision sensor and the proximity sensor, the first sensing signal is more suitable than the second sensing signal for generating the first portion of the environmental map, the second sensing signal is more suitable than the first sensing signal for generating the second portion of the environmental map, and the first portion is different from the second portion;
generate a first set of depth images for the first portion of the environmental map based on the first sensing signal and a second set of depth images for the second portion of the environmental map based on the second sensing signal, the first set of depth images including a portion of the first depth information corresponding to the first portion of the environmental map and the second set of depth images including a portion of the second depth information corresponding to the second portion of the environmental map;
combine the first set of depth images and the second set of depth images to generate the environmental map; and
effect the UAV to navigate in the environment using the environmental map.