US 11,810,376 B2
Method, apparatus and storage medium for detecting small obstacles
Yutong Zang, Beijing (CN)
Assigned to Beijing Xiaomi Intelligent Technology Co., Ltd., Beijing (CN)
Filed by Beijing Xiaomi Intelligent Technology Co., Ltd., Beijing (CN)
Filed on May 9, 2020, as Appl. No. 16/870,966.
Claims priority of application No. 201911135772.5 (CN), filed on Nov. 19, 2019.
Prior Publication US 2021/0149411 A1, May 20, 2021
Int. Cl. G06V 20/64 (2022.01); G06T 7/73 (2017.01); G05D 1/02 (2020.01); G06V 20/58 (2022.01); G06F 18/2413 (2023.01); G06V 20/10 (2022.01)
CPC G06V 20/64 (2022.01) [G05D 1/027 (2013.01); G05D 1/0221 (2013.01); G05D 1/0238 (2013.01); G05D 1/0251 (2013.01); G06F 18/24147 (2023.01); G06T 7/74 (2017.01); G06V 20/10 (2022.01); G06V 20/58 (2022.01); G05D 2201/0203 (2013.01); G06T 2207/10028 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A method for detecting small obstacles, applied to a cleaning robot and being implemented by a processor in the cleaning robot, the method comprising steps of:
acquiring a first 3D point cloud corresponding to image data acquired by the cleaning robot and first inertial measurement unit data collected by one or more sensors of the cleaning robot;
extracting, from the first 3D point cloud, a second 3D point cloud of a ground region;
extracting, from the second 3D point cloud, a third 3D point cloud having a height value in a set height range, wherein the set height range is a range containing a positive and/or a negative range;
calculating a ground projection point cloud of the third 3D point cloud;
determining morphologically-connected regions in the ground projection point cloud; and
using a morphologically-connected region having an area less than a preset value as a region where a small obstacle is located;
wherein an approach to acquire 3D point cloud data contains: an RGB depth image is acquired by an RGBD camera arranged on the cleaning robot, a depth image is extracted from the RGB depth image, and coordinate transformation is performed on the depth image to obtain the first 3D point cloud; or the first 3D point cloud of the cleaning robot is remotely received through wireless communication;
wherein the extracting, from the first 3D point cloud, a second 3D point cloud of a ground region comprises:
obtaining plane information by one of the following:
(i) searching, in a mapping set, second inertial measurement unit data closest to the first inertial measurement unit data; determining plane information corresponding to the second inertial measurement unit data according to the mapping set; wherein the mapping set is constructed according to historical second inertial measurement unit data of the cleaning robot during operation and corresponding plane information, and the mapping set comprises a mapping relationship between the historical second inertial measurement unit data and the plane information; or
(ii) training a learning model by using a training data set to obtain a first model;
wherein training input data and training output data in one-to-one correspondence in the training data set are respectively inertial measurement unit data and plane information corresponding to ground data in a 3D point cloud acquired by the cleaning robot at a same historical moment, respectively;
inputting the first inertial measurement unit data into the first model to obtain plane information output by the first model;
wherein the extracting, from the first 3D point cloud, a second 3D point cloud of a ground region further comprises:
extracting, from the first 3D point cloud, a second 3D point cloud of a ground region, according to the plane information;
wherein the plane information is used for indicating a plane in a world coordinate system.