| CPC G06V 10/95 (2022.01) [B64C 39/024 (2013.01); G06T 7/20 (2013.01); G06V 20/17 (2022.01); H04N 23/661 (2023.01); H04N 23/667 (2023.01); H04W 4/029 (2018.02); B64U 2101/30 (2023.01); G06T 2207/10032 (2013.01)] | 3 Claims |
|
1. A system for real-time target identification of an unmanned aerial vehicle (UAV) based on an embedded technique and an improved You Only Look Once version 4 (YOLOv4) algorithm, comprising a UAV client, a UAV management server, and a UAV management front end, wherein
the UAV client comprises a target identification client, a change detection client, and a target tracking client, wherein the target identification client comprises an image acquisition module, a target identification module, a storage module, and a file transfer module; the image acquisition module acquires position information and image data under vision of a UAV, the position information is acquired by a global positioning system (GPS) module, the image data is acquired through automatic photographing or manual photographing, the automatic photographing is completed after a preset photographing interval, and the manual photographing is to photograph after a photographing instruction from the UAV management server is received;
the position information and the image data obtained by the image acquisition module are transmitted to the target identification module, and the target identification module identifies a target and stores an identification result in the storage module; the file transfer module regularly scans the storage module to determine whether the identification result has been transmitted, encodes, if the identification result is not transmitted, the identification result in the storage module, converts an encoded identification result into a JavaScript Object Notation (JSON) data format, and transmits the data to a Netty file transfer service module; and when transmission is idle, the target identification client regularly sends a heartbeat signal to the UAV management server, to confirm whether the target identification client is normally connected to the UAV management server;
the change detection client comprises an image acquisition module, a change detection module, a storage module, and a file transfer module; the image acquisition module acquires position information and image data under vision of the UAV, the position information is acquired by the GPS module, the image data is acquired through automatic photographing or manual photographing, the automatic photographing is completed after a preset photographing interval, and the manual photographing is to photograph after a photographing instruction from the UAV management server is received; the position information and the image data obtained by the image acquisition module are transmitted to the change detection module, and the change detection module detects an image change and stores a detection result in the storage module; the file transfer module regularly scans the storage module to determine whether the detection result has been transmitted, encodes, if the detection result is not transmitted, the detection result in the storage module, converts an encoded detection result into a JSON data format, and transmits the data to a Netty file transfer service module; and when transmission is idle, the change detection client regularly sends a heartbeat signal to the UAV management server, to confirm whether the change detection client is normally connected to the UAV management server;
the target tracking client comprises an image acquisition module, a target tracking module, a storage module, and a file transfer module; the image acquisition module acquires position information and image data under vision of the UAV, the position information is acquired by the GPS module, the image data is acquired through automatic photographing or manual photographing, the automatic photographing is completed after a preset photographing interval, and the manual photographing is to photograph after a photographing instruction from the UAV management server is received; the position information and the image data obtained by the image acquisition module are transmitted to the target tracking module, and the target tracking module determines a target and stores a determining result in the storage module; the file transfer module regularly scans the storage module to determine whether the determining result has been transmitted, encodes, if the determining result is not transmitted, the determining result in the storage module, converts an encoded determining result into a JSON data format, and transmits the data to a Netty file transfer service module; and when transmission is idle, the target tracking client regularly sends a heartbeat signal to the UAV management server, to confirm whether the target tracking client is normally connected to the UAV management server;
the UAV management server comprises a Netty file transfer service module and a UAV management module, and the UAV management module comprises UAV management, information management, and photographing management; the Netty file transfer service module receives data transmitted by the UAV client and then decodes and restores the data, and then transmits the data to the UAV management module; the UAV management module performs information management on the received results and position information data, creates an “identification result folder”, a “detection result file”, and a “determining result folder”, and stores corresponding results and position information into corresponding folders; the Netty file transfer service module detects a heartbeat sent by the UAV client and feeds back a current connection state, the heartbeat detection result is transmitted to the UAV management, and online information and last-heartbeat interaction information of the client are stored to the UAV management; the Netty file transfer service module implements automatic connection between the client and the server after a network of the UAV client and the UAV management server is interrupted and then restored; when a network interruption occurs during file transfer, the Netty file transfer service module implements continued transmission; after receiving a “manual photographing” request from the UAV management front end, the photographing management in the UAV management module sends a photographing instruction to the Netty file transfer service module, and the Netty file transfer service module converts the photographing instruction into a Json data format, and transmits the data to the image acquisition module by using a heartbeat, to implement manual photographing; and
the UAV management front end is used to implement UAV information search, picture display, and control of manual photographing; and the UAV management front end searches a client of interest by using an IP address, to obtain online information, last-heartbeat interaction information, and results and position information data of the client, and displays the results and position information data of the client in pictures;
wherein the improved YOLOv4 algorithm is implemented by the following steps:
obtaining images of a target object under vision of the UAV, and marking the obtained images by using a marking tool, to obtain annotation files and annotation pictures;
performing anchor boxes clustering on the annotation files to obtain execution files;
modifying configuration files based on to a quantity of to-be-trained targets, and training a model based on the annotation files and the annotation pictures by using the execution files to generate weight files;
accelerating reasoning on the trained model, the execution files, and the generated weight files using a Tensor Runtime (TensorRT) accelerating reasoning device to obtain a reasoning model and an engine file; and
loading the reasoning model and the engine file for high-speed and high-precision target identification.
|