US 12,105,199 B2
System and method for measuring image distance of power transmission lines with unmanned aerial vehicle (UAV)
Yongdong Dai, Jiangsu (CN); Jianguang Yao, Jiangsu (CN); Feng Mao, Jiangsu (CN); Maofei Wang, Jiangsu (CN); Wanjin Yu, Beijing (CN); Beibei Weng, Jiangsu (CN); Ling Ju, Jiangsu (CN); Zhongjun Jiang, Jiangsu (CN); Xinlian Bu, Jiangsu (CN); Weihao Fan, Jiangsu (CN); Ze Zhang, Jiangsu (CN); and Xingchun Xu, Jiangsu (CN)
Assigned to State Grid Jiangsu Taizhou Power Supply Company, Taizhou (CN); and Allcorehatress (Beijing) Technology Co., Ltd., Beijing (CN)
Appl. No. 17/765,849
Filed by State Grid Jiangsu Taizhou Power Supply Company, Jiangsu (CN); and Allcorehatress (Beijing) Technology Co., Ltd., Beijing (CN)
PCT Filed Dec. 30, 2021, PCT No. PCT/CN2021/143374
§ 371(c)(1), (2) Date Apr. 1, 2022,
PCT Pub. No. WO2022/170878, PCT Pub. Date Aug. 18, 2022.
Claims priority of application No. 202110177572.7 (CN), filed on Feb. 9, 2021.
Prior Publication US 2024/0053477 A1, Feb. 15, 2024
Int. Cl. G01S 17/86 (2020.01); B64U 10/13 (2023.01); B64U 101/00 (2023.01); G01S 17/42 (2006.01); G01S 17/894 (2020.01); G01S 17/933 (2020.01); G06T 7/73 (2017.01); G06V 20/52 (2022.01)
CPC G01S 17/86 (2020.01) [G01S 17/42 (2013.01); G01S 17/894 (2020.01); G01S 17/933 (2013.01); G06T 7/73 (2017.01); G06V 20/52 (2022.01); B64U 10/13 (2023.01); B64U 2101/00 (2023.01); G06T 2207/10028 (2013.01)] 8 Claims
OG exemplary drawing
 
1. A system for measuring an image distance of power transmission lines with an unmanned aerial vehicle (UAV), wherein the power transmission lines comprise a plurality of electrical power towers and a power transmission line corridor between adjacent electrical power towers, and the system comprises a UAV, a master console, and a plurality of cameras;
the cameras are fixed on the electrical power towers, and are configured to collect two-dimensional (2D) image data of the power transmission line corridor;
the UAV comprises a vehicle body, a lidar device, a rotor assembly, a power unit, a flight control processor, a state detection device, and a wireless communication module;
the state detection device is configured to acquire positioning information of the state detection device, collect distance information between the UAV and ground, and send the positioning information and the distance information to the flight control processor; the flight control processor is configured to generate a control signal according to the distance information and the positioning information, and control, according to the control signal, the power unit to adjust a flight track and a flight height of the UAV; the lidar device is configured to collect three-dimensional (3D) point cloud data of the power transmission lines, wherein the 3D point cloud data is sent to the master console through the wireless communication module;
the master console is configured to receive the 2D image data and the 3D point cloud data, and establish a mapping relationship between the 2D image data and the 3D point cloud data according to interior orientation elements of the camera and exterior orientation elements of the 2D image data in a 3D space; recognize a target object according to the 2D image data, determine 3D point coordinates of the target object according to the mapping relationship, and calculate a distance from the target object to the power transmission lines according to the 3D point coordinates of the target object; and
after recognizing the target object according to the 2D image data, the master console is further configured to perform the following operations:
framing the target object of the 2D image data with a rectangular shape, and obtaining pixel coordinate values of any two diagonal corners of the rectangular shape;
calculating a pixel coordinate value of another corner at the bottom of the rectangular shape according to the pixel coordinate values of the two diagonal corners, obtaining 3D point coordinates corresponding to two base corners at the bottom of the rectangular shape according to the mapping relationship, and calculating a Euclidean spatial distance between the two base corners; wherein the mapping relationship is as follows:

OG Complex Work Unit Math
wherein dx and dy represent physical dimensions of each pixel on a horizontal axis x and a vertical axis y of the 2D image data, (u0, v0) is pixel coordinates of an intersection between an optical axis of the camera and a 2D image data plane, f represents a focal length of the camera, (u, v) is 2D pixel coordinates, and (XW, YW, ZW) is 3D space coordinates; R represents a rotation matrix between a 2D pixel coordinate system and a 3D space coordinate system, and T represents a translation vector from the 2D pixel coordinate system to the 3D space coordinate system; and the interior orientation elements of the camera comprise the pixel coordinates of the intersection between the optical axis of the camera and the 2D image data plane and the focal length of the camera;
dividing the Euclidean spatial distance between the two base corners by a pixel distance between the two base corners, obtaining a space size corresponding to each pixel of the rectangular shape, and calculating a spatial distance value between a top corner of the rectangular shape and the base corner on the same side;
on the basis of the 3D point coordinates of the two base corners, adding the spatial distance value on the Z axis of the 3D space coordinate system, to obtain 3D point coordinates of two top corners of the rectangular shape; and
calculating Euclidean spatial distances from the two top corner to a 3D point of a closest power transmission line according to the 3D point coordinates of the two top corners respectively, and using a smaller distance value as a distance from the target object to the power transmission lines.