Research on efficient grasping of unmanned aerial vehicle robotic arm based on visual servoing technology in transmission line inspection operations

With the continuous expansion of the application fields of contemporary UAVs, it is of practical significance and application demand to add a robotic arm to the UAV platform to enable it to have the active operation capability in complex environments. Compared with other robots, UAV robotic arms have the following advantages: first, they can realize the grasping of targets on the ground or in the air, such as performing sample collection tasks in the field of unmanned scientific research and environmental testing, and second, they can quickly reach the environment inaccessible to ground robots, and perform fine tasks such as installing or recovering measurement equipment [1]. Target autonomous grasping is one of the most typical applications of UAVs, and the target grasping task contains two core technologies: target perception technology and UAV robotic arm planning and control technology. In terms of target state perception technology, machine vision technology has become one of the most important and widely used perception methods in the field of UAVs with its advantages of wide application range, high perception accuracy, rich information, and non-contact [2]. The UAV robotic arm based on visual servo technology formed by the combination of vision technology and UAV robotic arm control technology is the most mature technology applied in the field of grasping at present. Through visual perception of the environment around the UAV robotic arm, the relative position between the target and the UAV robotic arm is solved, and on this basis, the position control or trajectory tracking control of the UAV robotic arm is realized, which in turn guides the UAV robotic arm system flight maneuvering to track the target for grasping [3].

Electricity production is the most important factor in the development of the country as it is the most consumed energy source in the world. Electrical energy is mainly transported through high-voltage and high-pressure transmission lines, which cross thousands of kilometers through mountain ranges, forests, and off-the-beaten-path areas to deliver electrical energy to where it is needed. Most transmission lines are exposed in wild and sparsely populated areas, and foreign objects hanging from them can cause great safety hazards [4]. In order to ensure proper operation of transmission lines, they need to be inspected periodically to keep track of the status of control cables, directional towers, and accessories, and the surroundings need to be inspected to eliminate all risks that may affect the good transmission of electrical energy. However, due to the large number of cable towers, the huge kilometers of transmission lines, and the remote locations and rugged terrain in which some lines are erected, cost, time, and personal safety are the main criteria to be followed by operators when selecting a method of power inspection [5]. In order to ensure that the transmission equipment is in proper condition, it is necessary to locate current problems detected and observe things that may cause problems in the future, such as nearby human living areas and new plants or bird nests, at present, inspections are mainly carried out by inspectors, which can lead to a number of problems, the most important of which is the wastage of time, as it takes a whole day just to inspect one pylon, which results in a high cost of manpower. In addition to this, inspection is difficult as most transmission lines are located in remote and difficult areas. Manual inspection requires people to the scene, but most of the time the environment is complex, need to go over the mountains, and the general tower thirty to fifty meters high, like the extra-high voltage tower hundreds of meters high, it is difficult for people on the ground to see clearly, and sometimes there will be trees to cover, the need to climb the tower, electric work walk the line to inspect defects, there is a very high risk of safety [6]. Therefore, the use of UAV robotic arm in the inspection operation of high-voltage transmission lines has multiple advantages that cannot be achieved by human labor, and can overcome the above existing problems. Its different advantages and its ability to replace humans in hazardous environments make it an ideal choice for various application scenarios [7].

In this paper, through the D-H matrix, the tandem-type mechanical arm is modeled, and the spatial transformation of the adjacent coordinate system is described through the spatial transformation matrix to get the position of the coordinate system, and the kinematic equation of the mechanical arm is deduced. The Lagrange equation is used to analyze the kinetic energy and potential energy of the mechanical arm in combination with the characteristics of the mechanical arm to complete the construction of the kinetic equation of the mechanical arm. In order to avoid the problem of the UAV robotic arm colliding with the camera during the grasping task, this paper uses the on-board-eye-out-of-hand configuration to form the UAV vehicle on the basis of the visual servo system. Based on the camera's position under the current space, the visual servo control of the UAV is realized. The grasping strategy of the flying robotic arm is set with reference to the predatory action of the bald eagle. Build the visual servo flying robotic arm designed in this paper on the UAV, and use the hooking method to deal with the grasping problem. Simulation experiments are carried out to verify the grasping effect of the UAV robotic arm in transmission line inspection, and its index elements such as movement speed and position estimation accuracy are analyzed.

2

Overview

2.1

UAV robotic arm based on visual servo technology

With the continuous expansion of contemporary UAV application fields, it is of practical significance and application demand to add a robotic arm to the UAV platform to enable it to have the ability of active operation in complex environments. Literature [8] designed an unmanned aerial vehicle (UAV) with a robotic arm aimed at handling tasks that cannot be accomplished by humans and verified the effectiveness of this approach through actual flight tests, which provides technical support for remote sensing and sampling applications without affecting flight performance. Literature [9] designed a visual servo-based aerial manipulator system based on the original aerial manipulator system using a separated control strategy to realize the target grasping by the aerial manipulator. Literature [10] proposed a practical visual servo control method using a spherical projection model and applied it to a UAV equipped with a robotic arm, aiming to improve the ability of the UAV robotic arm to grasp the target object, and the feasibility of the proposed method was verified through simulation tests. Literature [11] designed a fault-tolerant visual servo control system for a robotic arm by introducing a state observer based on radial basis function neural network and proposing a fault-tolerant controller based on non-vectorized fast terminal sliding modes, and verified the effectiveness and robustness of the designed system through numerical simulations.

In addition, literature [12] proposed a novel image visual servo controller based on natural features rather than artificial markers and applied it to a six-degree-of-freedom (6-DoF) aerial manipulator system in order to realize autonomous aerial grasping of target objects, and verified the practicality of the proposed method through experiments, which provides technical support for the research of aerial transportation and manipulation in the field of unmanned aerial vehicles (UAVs). Literature [13] proposes an uncalibrated image-based visual servoing strategy for UAV manipulation, and the effectiveness of the proposed strategy in aerial manipulation is verified through simulation and real experiments, which improves the flight behavior of the UAV and avoids the problem of restricted joints of the UAV robotic arm. Literature [14] analyzes the stability of the proposed image-based visual servo control strategy through Liapunov's theory and illustrates the performance of the proposed method in tracking and grasping a moving target in an unmanned aerial manipulator system through comparative simulations. In the literature [15], in order to realize the grasping planning and visual servoing of outdoor aerial dual manipulator, a system for grasping known objects using a dual manipulator UAV equipped with an RGB-D camera is designed using artificial neural networks, alignment algorithms, Kalman filters, and a three-dimensional (3D) model of the object, and experimentally verified to have a feasibility.

2.2

The application of UAV robotic arm in power transmission line inspection operation

On the basis of ensuring the excellent performance of the drone mechanical arm, it is applied to power transmission line inspection, which can accurately and quickly find the fault point and provide help for fault handling. Literature [16] examined the application of intelligent aerial robots in the inspection and maintenance of power lines, the main content of which is based on the aerial cognitive integrated multitasking robotic system with extended operating range and safety developed methods and technologies such as application of perceptual model predictive control, 3D solid-state LIDAR and RGB camera remote mapping methods, and UAV robotic arm, and used these technologies for inspection and maintenance of power lines. Literature [17] designed a compliant lightweight robotic arm with a special end-effector and mounted it on a UAV in order to facilitate a wide range of inspection and maintenance tasks on transmission lines, which in turn ensures the safe operation of transmission lines. Literature [18] synthesized the point cloud-based foreign object localization method and hierarchical task priority control method to propose a position-based visual servoing technique and applied it to the UAV robotic arm, and verified the effectiveness of the proposed method through the foreign object removal experiments on transmission lines, which can enable the UAV robotic arm to accurately grasp in hazardous environments.

3

UAV robotic arm model construction

3.1

Drone model

3.1.1

Kinematic modeling of drone robotic arms

In order to have the same standard, for the modeling of tandem type robotic arm researchers often use D-H matrix for modeling [19]. D-H matrix has four parameters, its main role is to add the reference coordinate system to the connecting rod of the robotic arm, and then through the spatial transformation matrix to describe the spatial transformation of the neighboring coordinate system, through the spatial transformation to get the tail of the robotic arm with respect to the base coordinate system of the position, and then you can derive the kinematic model of the robotic arm. In this project, when modeling the robotic arm by applying the D-H method, the coordinate system is determined firstly, the center point of each joint is O_i(i = 0,1,2), the axis of rotation of the joint is Z_i (i = 0,1,2), the common perpendicular of Z_i and Z_i+1 is X_i(i = 0,1,2), and finally the Y_i(i = 0,1,2) axis is determined according to the right-hand rule, and the coordinate system established according to the above method is shown in Fig. 1, and in particular, the center point of the coordinate is O_o and O₁ coincide with each other.

l₁ denotes the length of the first linkage of the robotic arm, and θ₁ denotes the angle of rotation of joint 1. According to the rules of the D-H modeling method, the spatial transformation relationship of the proximity coordinate system can be deduced as: 1

_{i}^{i - 1} T = [\begin{matrix} d θ_{i} & - s θ_{i} & 0 & α_{i - 1} \\ s θ_{i} c α_{i - 1} & c θ_{i} c α_{i - 1} & - s α_{i - 1} & - s α_{i - 1} d_{i} \\ s θ_{i} s α_{i - 1} & c θ_{i} s α_{i - 1} & c α_{i - 1} & c α_{i - 1} d_{i} \\ 0 & 0 & 0 & 1 \end{matrix}]

Next, the spatial transformation matrix from the base coordinate system to the robotic arm linkage coordinate system is derived from Eq. (1) as $_{1}^{0} T$ , and the spatial transformation matrix from the robotic arm linkage coordinate system to the tail actuator coordinate system is $_{2}^{1} T$ . The specific formulas for the two transformation matrices can be shown as: 2 $_{1}^{0} T = [\begin{matrix} c θ_{1} & - s θ_{1} & 0 & 0 \\ s θ_{1} & θ_{1} & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix}],_{2}^{1} T = [\begin{matrix} 1 & 0 & 0 & l_{1} \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix}]$

From the equation (2)'s it can be deduced that the end fixture to base transformation matrix is $_{2}^{0} T$ : 3 $_{2}^{0} T =_{1}^{0} T_{2}^{1} T = [\begin{matrix} c θ_{1} & - s θ_{1} & 0 & l_{1} c θ_{1} \\ s θ_{1} & c θ_{1} & 0 & l_{1} s θ_{1} \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix}]$

Then the position of the end gripper of the robot arm in the base coordinate system is defined as $P_{j}^{0}$ and can be expressed as: 4 $P_{j}^{0} = [\begin{matrix} x_{j}^{0} \\ y_{j}^{0} \\ z_{j}^{0} \\ 1 \end{matrix}] = [\begin{array}{l} 0 \\ 0 \\ 0 \\ 1 \end{array}] = [\begin{matrix} l_{1} c θ_{1} \\ l_{1} s θ_{1} \\ 0 \\ 1 \end{matrix}]$

According to Eq. (3) and Eq. (4), defining $_{J_{0}}^{B} T$ as the spatial rotation matrix of the robotic arm base coordinate system J to the UAV coordinate system B, the synthesis can be obtained that the position of the robotic arm tail fixture in the world coordinate system can be expressed as $P_{n}^{3}$ , which can be demonstrated as: 5 $P_{n}^{3} = R_{B J_{0}}^{W B} T_{1}^{0} T_{2}^{1} T P_{j}^{0}$

3.1.2

Robotic arm dynamics model

There are mainly Newton–Euler method and Lagrange method for modeling the dynamics model of the robotic arm, and in this project, the Lagrange method is used to analyze the kinetic and potential energy of the robotic arm in combination with the characteristics of the robotic arm, and construct the dynamics equation of the robotic arm [20]. It is known that the Lagrange equation can be expressed in the following component form: 6 $\frac{d}{d_{i}} \frac{\partial L}{\partial {\dot{θ}}_{i}} - \frac{\partial L}{\partial {\dot{θ}}_{i}} = u_{i}$ where u_i denotes the ind moment and θ_i denotes the angle of rotation of the arm.

The Lagrange equation is [21]: 7 $L = K - U$ where X is the kinetic energy of the system under study and U is the potential energy of the system under study.

Defining the zero potential energy surface as the X -axis counterpart, the kinetic energy of the robotic arm linkage 1 is calculated as: 8 $K_{1} = \frac{1}{2} m_{1} l_{1}^{2} {\dot{θ}}_{1}^{2}$

The connecting rod 1 potential energy can be expressed as: 9 $U_{1} = - m_{1} g_{l} l_{1} \cos θ_{1}$

The potential energy of the base of the robotic arm is at the zero potential energy surface its value is 0 and the kinetic energy is: 10 $K_{2} = \frac{1}{2} m ({\dot{x}}_{p}^{2} + {\dot{z}}_{p}^{2}) + \frac{1}{2} J_{y y} q^{2}$

Finally the overall kinetic and potential energy of the system can be obtained as: 11 ${\begin{array}{l} C = K_{1} + K_{2} \\ U = U_{1} \end{array}$

Then finally the robotic arm dynamics equation is obtained according to equation (11).

3.2

Visual servo framework for flying robotic arms

3.2.1

Definition of visual servo system coordinate system

The visual servo system of the flying robotic arm in this project is a heterogeneous eye-in-hand structure, which is referred to as the onboard-eye-in-hand configuration, and its name is also used in this project. The advantage of this approach is that it can avoid the problems of excessive image perturbation caused by the camera mounted on the actuator at the end of the flying robotic arm, or collision of the camera with the object to be grasped during the grasping task. Moreover, this configuration also solves the situation that the target object is lost in the field of view of the camera due to the insufficient field of view of the camera when the angle of movement of the robotic arm is too large.

So the on-board-eye-on-hand configuration hand-eye calibration of this topic can be divided into three parts. The first part is the camera calibration to get the camera pixel coordinate system to camera coordinate system conversion relationship. The second part is the camera-IMU calibration to get the camera coordinate system to IMU coordinate system conversion relationship. The third part is to use the kinematic model in subsection 2.1.1 to obtain the matrix transformation from the IMU coordinate system to the robot arm end coordinate system. As expressed in equation (12) as: 12 $T_{P i x e d}^{H a n d} = T_{I M U}^{H a n d} T_{C a m e r a}^{I M U} T_{P i x e d}^{C a m e r a}$ where $T_{P i x d e}^{H a n d}$ denotes the transformation matrix of pixel coordinates to the end fixture of the flight console, $T_{P i v e r}^{C a n e r e}$ denotes the rotation matrix of pixel sits, which can be derived from the camera-IMU calibration, and $T_{I M U}^{H a n d}$ denotes the transformation relationship from the IMU coordinate system to the tail coordinates of the actuator of the flight console, assuming that the IMU coordinate system is exactly overlapping with the robotic arm base coordinate system, which is equivalent to the transformation matrix of the robotic arm base coordinate system to the tail of the actuator of the flight console. The transformation matrix of the robot arm base coordinate system to the flight console actuator tail coordinate system.

3.2.2

Position-based visual servo control

In the flying robotic arm system designed in this topic, the UAV will be restricted to fly at a fixed altitude, i.e., the altitude of the flying robotic arm system in space is known. In the case that the flight height is known, this topic will set a fixed point in the grasping scene as the grasping detection point of the flying robotic arm, so that the coordinates derived from each pixel point in the camera can be converted to the real scale by the camera internal reference.

In the position-based visual servo control system, it is necessary to know the position of the camera coordinate system relative to a certain coordinate system in the current space as an input to the system s. In this project, the center point of the frame to be grasped through the detection of the target to be grasped as the origin of the coordinates to establish a coordinate system F_c^⁎. so that the camera coordinate system is F_c, then it can be obtained: 13 $s = {(c^{*} t_{c}, θ u)}^{T}$ where c^⁎t_c is the coordinate representation of the camera coordinate system origin in the coordinate system F_c^⁎, and θu in equation (13) represents the rotation of the camera coordinate system F_c with respect to the target coordinate system F_c^⁎.

Since the flying robotic arm is flying at fixed height, its position control is only carried out in the plane where the fixed height is located, so the UAV is only involved in translational motion as well as yaw motion, and the camera assembly position is known, and the camera coordinate system can be converted to the UAV coordinate system through the camera-IMU calibration. This subject is known to fly at an altitude of h, the camera focal length can be calibrated to f, then c^⁎t_c can be converted to the UAV relative to the target coordinate system coordinates through the equation (12) in $T_{C a n e r a}^{I M U}$ , that is, for the end of the robotic arm relative to the target coordinate system coordinates so that it is ^{^⁎t_j}: 14 $c^{*} t_{j} = T_{I M U}^{H a n d} T_{C a m e r a}^{I M U} c^{*} t_{c}$

In this project, the position control of the flying robotic arm is controlled using the existing flight control system Pixhawk 2.4.8APM flight control, which is actually PID control, and its desired input is the UAV arriving at F_c^⁎. The coordinate origin, and the feedback input is coordinate c^⁎t_j, which is directly above the target to be grasped when the control of the end of the flying robotic arm arrives at the coordinate origin. However, the attitude of the end of the flying robotic arm is different from the target to be grasped, and then the yaw of the UAV is adjusted by the θu in the formula (13), so that the end of the flying robotic arm actuator can be made to have the same attitude as the target to be grasped, and then the fixed height h can be lowered to complete the grasping task of the position-based visual servo flying robotic arm system.

3.2.3

Flying Robotic Arm Gripping Strategy

The grasping strategy of the flying robotic arm is to develop a plan for the flying robotic arm to approach the target and perform the grasping operation while ensuring the stable flight of the system. A good strategy will significantly improve the success rate and efficiency of the grasping task. The current researchers have a variety of grasping strategies, and this project mainly refers to the predatory behavior of bald eagles in the biological world to set up the grasping strategy of the flying robotic arm.

Referring to the grasping process of the bald eagle, this paper also divides the grasping task into three stages, but considering the actual task execution, this paper simplifies the process. The overall schematic is shown in Figure 2. In the first stage, the flight robot arm flies to the target detection area from the starting point at a specified height and hovers, detecting the current area to be grasped after the target grasping prediction. In the second stage, after predicting the grasping detection frame, the flying robot arm flies to the target directly above and adjusts the attitude according to the grasping detection result, and then descends to the specified altitude to the area to be grasped for grasping operation. In the third stage, the flying robot arm receives a successful signal and flies away from the area to be grasped to the end area to complete the grasping operation.

4

Simulation experiment verification

4.1

Gripping Program Design

The grasping scheme designed in this paper mainly includes three operating heads: UAV, connecting rod, insulating rod, control box and cutting assembly, screw hook assembly and gripper assembly. The drone adopts the DJI Matrice 610 drone, which has a load capacity of more than 8 kg and a endurance of 30 minutes. The visual servo flight manipulator designed in this paper is built on the UAV, and the grabbing method can be considered when the floating foreign objects on the distribution line are difficult to deal with by hooking. The gripper assembly has two states of “open” and “closed”, and the conversion of the “open” and “closed” states of the gripper assembly is effectively completed through the battery, geared motor, tie rod and other devices on the insulated operating rod.

4.2

Analysis of the effect of drone robotic arm in transmission line inspection grasping

4.2.1

Simulation experiment results

In order to verify the correctness of the designed visual servo control law using the inspection robot model established in Robotics Toolbox and the camera model established in Machine Vision Toolbox, a wire grabbing control simulation model is established in Simulink. According to the position of the camera and the imaging principle, when the transmission line is imaged in the horizontal middle position in the image, Fig. 3 shows the simulation of the wire-grabbing control, Fig. (a) shows the pixel adjustment trajectory, Fig. (b) shows the pixel deviation, and Fig. 1-8 shows the simulation experiments from 1 to 8 times.

As shown by the asterisk in Fig. 3(a), it indicates that the line is aligned to the transmission line, which has the desired coordinates:

uv_d = [78.847,289.734;73.905,164.027;412.409,288.603;470.803,253.535]^T. Assume that the image coordinates of the transmission line imaged at the initial deviation are:

uv_i = [84.854,219.457;94.891,82.579;416.058,211.538;489.051,164.027]^T, as shown by the cross in Fig. 3(a). Fig. 3(b) shows the pixel deviation of the robot grasping the line with time under the designed control law, from which it can be seen that the final pixel deviation converges to 0 after 10s, indicating the completion of the grasping line the simulation results show the feasibility of the image-based visual servo control method.

4.2.2

Wire Grabbing Control Experiment

The results of 12 more typical experiments (3rd, 5th, …, 99th) were selected from hundreds of experiments, and Table 1 shows the results of the line-grabbing control experiments, in which the larger values of termination bias are -22.485 pixels and 11.822 pixels, and the others are less than 2 pixels, and the termination angle of bias is 1.578° at the maximum, which are all in line with the tolerance range of the design. When the initial bias distance and bias angle are large (|θ(t)| > 15°, |λ(t)| > 180 pixel), it takes 120s to complete the line grabbing task. The longest time taken in the experiment was 240 s. The light in the laboratory is relatively low in the afternoon, but the wire grabbing task was finally accomplished. In the actual automatic obstacle-crossing, the traveling wheel deflection angle is |θ(t)| < 10°, the offset is |λ(t)| < 90 pixels, and it takes less than 40s to complete the line grasping task. From the experimental results, it can be seen that the visual servo-based line grasping method is accurate and reliable, and has a high efficiency.

Table 1.

Line-grasping control experiment results

Experimental serial number	3	5	14	26	35	42
Initial deflection θ(t)/(°)	6	22.482	19	26.756	3	27.958
Initial deviation λ(t)/pixels	197.554	-187.669	164.158	71.598	207.694	-203.482
End deflection θ(t)/(°)	1.321	1	1.515	1.578	1.566	0
Termination offset λ₀/pixels	-0.248	1	1.836	-0.287	-0.795	1
Scratch time/s	116	124	137	147	108	119
Experimental serial number	53	55	61	70	98	99
Initial deflection θ(t)/(°)	6.485	1	4	35	8	-1.657
Initial deviation λ(t)/pixels	187.652	206	123.856	-91.632	-195	-148.565
End deflection θ(t)/(°)	1.251	1	1	1	1.265	1.526
Termination offset λ₀/pixels	0.578	1	-0.578	0	-22.485	11.822
Scratch time/s	126	98	67	159	46	43

4.2.3

Random speed of motion in three dimensions

Further, the motion state of the target is extended to the three-dimensional space, so that the target moves in the Roll, Pitch and Yaw directions, in order to verify the accuracy and robustness of the tracking and grasping, and the motion of the target is divided into two scenarios of slow and fast during the experiment, and try to control the target's maximal linear velocity of about 0.1 m/s and the maximal angular velocity of about 30°/s in the case of slow, and in the case of fast Try to control the maximum linear velocity of the target is about 0.2m/s and the maximum angular velocity is about 60°/s. Repeat the test in each scenario.

The rapid random motion of the target in three-dimensional space is selected, and the data obtained in the experiment are plotted and analyzed.Fig. 4 shows the random motion velocity in three-dimensional space, Fig. 4(a) is the velocity in the Roll direction, Fig. 4(b) is the velocity in the Pitch direction, and Fig. 4(c) is the velocity in the Yaw direction. To reflect the tracking situation in all directions in an all-round way. From Figure 4, it can be seen that the target in the three-dimensional space from top to bottom to carry out the cyclotron movement, position and direction of the movement process have occurred sudden changes, the trajectory of the robotic arm and the target trajectory is relatively close to the target, there is a large error in the direction of the target after the sudden change in position, but the back is still able to keep up with. The velocity in each direction can be seen in the dotted line graph, the speed of the manipulator although there are fluctuations but on the whole with the change of the speed of the target changes, the maximum speed of the target during the movement of the tumbling, pitching, yawing is about 1.311rad/s, 0.825rad/s, 1.239rad/s.

4.3

Accuracy of Position Estimation for Physical Flight Robotic Arms

In the physical environment, the bit position provided by the motion capture system is taken as the true value and compared with the bit position obtained from the actual visual odometry estimation, and the experimental results are shown in Fig. 5. In the experimental scenario, the target is moving along a straight line, while the UAV is tracking the target from multiple directions. It can be seen that the estimated world position of the UAV is close to the true value, and in the x-axis, the true value of the UAV position has a high degree of overlap with the estimated trajectory, and only a small error occurs at 15s, 26s, and 32s, and the errors at these three time points are 0.013, 0.003, and 0.002m, respectively. In the z-axis, the error between the real value and the estimated value before 17s stays within 0.0015, and after 17s, the error gradually increases, and the maximum error value is close to 0.05 m. The absolute position error is larger than that of the simulation experiment. The reasons for the error mainly lie in the following points: 1)

In the actual environment, the body camera faces the wall, and the feature depth of the wall is generally greater than 5 m. The depth detection accuracy of the RealSense D435i at this distance is reduced.

2)

Due to the limited performance of the on-board NUC processor and the small WIFI transmission bandwidth, the visual odometer will drop frames during operation.

3)

The teeth of the physical robotic arm are larger, so the positive kinematic solution error of the robotic arm is larger, resulting in an increase in the estimation error of the world position of the target. Figure 5 shows the experiments of the position estimation accuracy of the physical flying robotic arm, and Figures (a), (b) and (c) are the x, y and z axes, respectively, which shows the position estimation accuracy of the physical neck-eye camera, and it can be seen that its estimation accuracy is high. This is due to the fact that in the tracking strategy of this paper, the camera is closer to the dynamic target in order to ensure the quality of the observation. This not only ensures the depth measurement accuracy of the RGB-D camera, but also increases the number of extracted features of the obtained target and reduces the probability of mis-matching, thus improving the position estimation accuracy. This is precisely the main advantage of the image vision servo-based active tracking strategy used in this paper. Even if the world position estimation error of the dynamic target is large, the tracking of the target can still be ensured by the visual servo tracking strategy. The 3D reconstruction of the target with visual servo grasping is only related to the position of the camera relative to the target, which ensures the observation quality of the target and can effectively improve the reconstruction accuracy.

5

Conclusion

In this paper, using the D-H matrix, the tandem type mechanical arm is modeled, and the coordinate system position information of the mechanical arm is obtained through the spatial variation, and the coordinate information of the mechanical arm is deduced. Then, the Lagrangian method is used to construct the mechanical arm dynamics equation by combining the characteristics of the mechanical arm. Define the spatial coordinates of the visual servo system to realize the position-based visual servo mechanical arm grasping control.

Build the visual servo flying robotic arm designed in this paper on the UAV, design the grasping scheme, and analyze the grasping effect of the UAV robotic arm in transmission line inspection through simulation experiments. The image coordinates of the desired coordinates and the assumed coordinates are uv_d = [78.847,289.734;73.905,164.027;412.409,288.603;470.803,253.535]^T, uv_i = [84.854,219.457;94.891,82.579;416.058,211.538;489.051,164.027]^T. Meanwhile, under the action of the designed control law, the pixel deviation of the robot grasping the line tends to be close to 0 after 10s, and the control method of the grasping action of the UAV robotic arm with the visual servo based on the image is feasible.

The more classic 12 experiments are selected from hundreds of experiments, in order to carry out the specific analysis of the grasping line control, the larger values of the termination deviation are -22.485 pixels and 11.822 pixels, and the others are all less than 2 pixels, and the termination deviation angle is 1.578° at most, which are all in line with the tolerance range of the design. In the automatic obstacle-crossing test, the UAV robotic arm designed in this paper completes the grasping using less than 40s, which has a high execution efficiency.

The random motion speed of the UAV in three-dimensional space is analyzed from three perspectives: tumbling, pitching, and yawing, and the maximum speeds of tumbling, pitching, and yawing of the target during the motion are about 1.311 rad/s, 0.825 rad/s, and 1.239 rad/s.

In the physical environment, the bit position provided by the motion capture system is taken as the true value and compared with the bit position obtained from the actual visual odometry estimation. In the x-axis, the trajectories of the true value and estimated value of the UAV position have a high degree of overlap, and only a small error occurs at 15s, 26s, and 32s, and the errors at these three time points are 0.013, 0.003, and 0.002m, respectively, which shows that the accuracy of the position estimation of the robotic arm in the physical flight state is more accurate.

Język:: Angielski

Częstotliwość wydawania:: 1 razy w roku
Dziedziny czasopisma:: Nauki biologiczne, Nauki biologiczne, inne, Matematyka, Matematyka stosowana, Matematyka ogólna, Fizyka, Fizyka, inne

Kanał RSS czasopisma

Research on efficient grasping of unmanned aerial vehicle robotic arm based on visual servoing technology in transmission line inspection operations

Jinfu Han

Songnan Liu

Peiyong Yu

Wei Yang

Xinlei Xu

Data publikacji: 24 wrz 2025

Otrzymano: 19 sty 2025

Przyjęty: 20 kwi 2025

DOI: https://doi.org/10.2478/amns-2025-0953

Słowa kluczoweD-H matrix, Lagrange equation, Flying robotic arm grasping, Vision servo framework

© 2025 Jinfu Han et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Słowa kluczowe
D-H matrix, Lagrange equation, Flying robotic arm grasping, Vision servo framework