| CPC G06T 7/74 (2017.01) [G01S 19/485 (2020.05); G05D 1/101 (2013.01); G06T 7/50 (2017.01); B64U 50/19 (2023.01); B64U 2101/30 (2023.01); B64U 2101/60 (2023.01); G06T 2207/10032 (2013.01); G06T 2207/20081 (2013.01)] | 19 Claims |
|
1. A method comprising:
receiving a two-dimensional (2D) image captured by a camera on a unmanned aerial vehicle (UAV) and representative of an environment of the UAV;
applying a trained machine learning model to the 2D image to produce a semantic image of the environment and a depth image of the environment, wherein the machine learning model has been trained with a semantics branch to produce the semantic image and a depth branch to produce the depth image, and wherein the semantic image comprises one or more semantic labels;
retrieving reference depth data representative of the environment, wherein the reference depth data includes reference semantic labels; and
determining a location of the UAV in the environment, wherein determining the location of the UAV in the environment comprises aligning the depth image of the environment with the reference depth data representative of the environment, wherein aligning the depth image of the environment with the reference depth data representative of the environment is based on associating the one or more semantic labels from the semantic image with the reference semantic labels from the reference depth data; and
controlling the UAV to navigate in the environment based on the determined location of the UAV in the environment.
|