US 12,141,985 B2
Weeding robot and method and apparatus for planning weeding path thereof, and medium
Longfei Cui, Jiangsu (CN); Xinyu Xue, Jiangsu (CN); Feixiang Le, Jiangsu (CN); Tao Sun, Jiangsu (CN); Songchao Zhang, Jiangsu (CN); Chen Chen, Jiangsu (CN); Yongkui Jin, Jiangsu (CN); Yang Xu, Jiangsu (CN); Zhu Sun, Jiangsu (CN); Suming Ding, Jiangsu (CN); Qingqing Zhou, Jiangsu (CN); Chen Cai, Jiangsu (CN); Wei Gu, Jiangsu (CN); and Wei Kong, Jiangsu (CN)
Assigned to NANJING INSTITUTE OF AGRICULTURAL MECHANIZATION, MINISTRY OF AGRICULTURE AND RURAL AFFAIRS, Jiangsu (CN)
Appl. No. 18/696,169
Filed by NANJING INSTITUTE OF AGRICULTURAL MECHANIZATION, MINISTRY OF AGRICULTURE AND RURAL AFFAIRS, Jiangsu (CN)
PCT Filed Apr. 21, 2022, PCT No. PCT/CN2022/088072
§ 371(c)(1), (2) Date Mar. 27, 2024,
PCT Pub. No. WO2023/050783, PCT Pub. Date Apr. 6, 2023.
Claims priority of application No. 202111147412.4 (CN), filed on Sep. 29, 2021.
Prior Publication US 2024/0265547 A1, Aug. 8, 2024
Int. Cl. G06T 7/11 (2017.01); A01B 39/18 (2006.01); A01B 69/04 (2006.01); G06T 7/521 (2017.01); G06V 10/764 (2022.01); G06V 20/10 (2022.01); G06V 20/70 (2022.01)
CPC G06T 7/11 (2017.01) [A01B 39/18 (2013.01); A01B 69/008 (2013.01); G06T 7/521 (2017.01); G06V 10/764 (2022.01); G06V 20/188 (2022.01); G06V 20/70 (2022.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 8 Claims
OG exemplary drawing
 
1. A method for planning a weeding path for a weeding robot, comprising:
acquiring an image segmentation model based on neural network model training, wherein the image segmentation model is configured to identify and segment a weed target feature, a soil target feature and a crop target feature; and
based on the weed target feature, acquiring the weed target feature by intra-camera tracking, and acquiring the weed target feature by inter-camera tracking, and planning a weeding path for a weeding robotic arm of the weeding robot, to enable the weeding robotic arm to weed according to the weeding path,
wherein a first camera, . . . , an i-th camera, . . . , an N-th camera are arranged sequentially at intervals in an opposite direction of a heading direction of the weeding robot, and the first camera to the N-th camera move with a movement of the weeding robot;
wherein based on the weed target feature, acquiring the weed target feature by intra-camera tracking, and acquiring the weed target feature by inter-camera tracking, and planning a weeding path for a weeding robotic arm of the weeding robot, comprises:
acquiring a first image captured by the first camera, identifying and segmenting the first image by the image segmentation model, to generate a first weed target feature and a weed label of the first weed target feature;
when the i-th camera is a second camera, acquiring an i-th image captured by the i-th camera, extracting a target feature in the i-th image according to a density-based spatial clustering method, and matching the target feature in the i-th image with an (i−1)-th weed target feature, to obtain an i-th weed target feature corresponding to the (i−1)-th weed target feature and a weed label of the i-th weed target feature; according to the i-th weed target feature and the weed label of the i-th weed target feature, planning the weeding path for the weeding robotic arm of the weeding robot;
when the i-th camera is a third camera to the N-th camera in sequence, extracting target feature in the i-th image according to the density-based spatial clustering method, matching the target feature in the i-th image with an (i−1)-th weed target feature to obtain an i-th weed target feature corresponding to the (i−1)-th weed target feature, and a weed label of the i-th weed target feature; according to the i-th weed target feature and the weed label of the i-th weed target feature, correcting the weeding path for the weeding robotic arm of the weeding robot;
wherein, N≥2, 1<i≤N, i and N are both positive integers; time for the N-th camera to move with the weeding robot to an original position of the first camera and a sum of time for image processing of the first camera to the N-th camera are greater than or equal to time for the weeding robotic arm of the weeding robot to swing to a position of a weed corresponding to the first weed target feature.