	US 11,967,161 B2
	Systems and methods of obstacle detection for automated delivery apparatus
Hakan Boyraz, Seattle, WA (US); Baoyuan Liu, Bellevue, WA (US); Xiaohan Nie, Lynnwood, WA (US); and Sheng Chen, Seattle, WA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Jun. 26, 2020, as Appl. No. 16/914,189.
Prior Publication US 2021/0405638 A1, Dec. 30, 2021
Int. Cl. G06T 7/73 (2017.01); B60W 60/00 (2020.01); G05D 1/00 (2006.01); G05D 1/02 (2020.01); G06K 9/00 (2022.01); G06V 10/80 (2022.01); G06V 20/10 (2022.01); G06V 20/58 (2022.01); G06V 20/64 (2022.01)

CPC G06V 20/64 (2022.01) [B60W 60/00256 (2020.02); G05D 1/0094 (2013.01); G05D 1/024 (2013.01); G05D 1/0274 (2013.01); G06T 7/74 (2017.01); G06V 10/80 (2022.01); G06V 20/10 (2022.01); G06V 20/58 (2022.01); G06T 2207/10024 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/30261 (2013.01)]

20 Claims

1. A system for comparing image information acquired by a delivery device, the system comprising:

an RGB sensor configured to provide context data;

a LIDAR sensor configured to provide range information;

a depth sensor configured to provide depth information;

a hardware processor configured to:

extract RGB features from the context data, LIDAR features from the range information, and depth features from the depth information;

combine the extracted features from at least two of the RGB sensor, the LIDAR, and the depth sensor to form environment data for the environment around the delivery device;

generate a terrain model based on the environment data, the terrain model including a plurality of terrain types;

detect an object based on the environment data;

generate, based at least in part on the environment data:

a dense point cloud and pose information output; and

3D pose and covariance matrix data;

combine the context data from the RGB sensor with the dense point cloud and pose information output;

generate object proposal data for the detected object and labeled point cloud data;

generate object states for the detected object; and

generate a map for an environment in which the delivery device operates based on the terrain model, the detected object, the generated object proposal data,

the labeled point cloud data, and the object states;

a program interface configured to provide access to generated information and the map of the environment of the delivery device; and

a navigation system configured to navigate the delivery device by processing the generated information and the map of the environment and determining a traversal path through the environment, wherein, during the navigation, the traversal path is determined based at least in part on traversability scores for individual terrain types of the plurality of terrain types, and wherein the traversability scores are measured based on traversability of each pixel or portion of the map relative to other pixels or portions of the map during the navigation.