CPC G05D 1/0238 (2013.01) [G05D 1/0223 (2013.01); G05D 1/0246 (2013.01); G06N 3/04 (2013.01); G06T 7/55 (2017.01); G06T 2207/10024 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30261 (2013.01)] | 20 Claims |
1. A carpet detection method, comprising:
obtaining, through an RGB-D camera, one or more RGB-D image pairs, wherein each RGB-D image pair comprises and RGB image and a depth image, and the depth image comprises data for representing a distance to scene objects in the depth image;
detecting one or more carpet areas in the RGB image of the one or more RGB-D image pairs and generating a first 2D bounding box to mark each of the one or more carpet areas using a first deep learning model based on a first data set containing a carpet class, wherein the first data set comprises a plurality of images of carpets in various scenes;
detecting one or more carpet-curl areas in the RGB image of the one or more RGB-D image pairs and generating a second 2D bounding box to mark each of the one or more carpet-curl areas using a second deep learning model based on a second data set containing a carpet-curl class, wherein the second data set comprises a plurality of images of carpet-curls in various scenes;
generating a group of carpet points corresponding to each of the one or more carpet areas in the RGB image of each of the one or more RGB-D image pairs by matching each pixel of the RGB image within the first 2D bounding box corresponding to the carpet area to each pixel in the depth image of the RGB-D image pair;
generating a group of carpet-curl points corresponding to each of the one or more carpet-curl areas in the RGB image of each of the one or more RGB-D image pairs by matching each pixel of the RGB image within the second 2D bounding box corresponding to the carpet-curl area to each pixel in the depth image of the RGB-D image pair; and
controlling a mobile machine to move and avoid one or more carpet-curls, according to the generated group of carpet points corresponding to each of the one or more carpet areas in the RGB image of each of the one or more RGB-D image pairs, and the generated group of carpet-curl points corresponding to each of the one or more carpet-curl areas in the RGB image of each of the one or more RGB-D image pairs.
|