Study on automatic target identification method in substation 3D scene model

The establishment of three-dimensional scene model of the substation is inseparable from the acquisition of three-dimensional point cloud data, and three-dimensional point cloud data acquisition relies on a variety of types of three-dimensional acquisition technology, this paper introduces the principle of three-dimensional point cloud data acquisition of LiDAR, to provide data support for the construction of three-dimensional scene model of the substation.

LiDAR is to measure the target distance, speed and position by transmitting a laser beam to the target and receiving the reflected light. It is mainly composed of laser, receiver, optical system, clock and data processing unit. The laser emits an infrared laser beam with high energy density and narrow beamwidth. The infrared laser beam is emitted to the target surface after passing through the beam control system, and the target will reflect the laser beam back and converge to the receiver through the optical system, which accepts the reflected light and converts it into electrical signals. The time difference of the reflected light is calculated by the clock recording the time of the laser beam from emission to reflection and then to the receiver, using the time difference of the reflected light to calculate the distance between the target and the LIDAR, the distance is calculated using the TOF time-of-flight method, which includes two kinds of pulse ranging and phase ranging.

Pulse ranging is through the measurement of laser pulses in the radar and the target between the back and forth flight time to obtain information on the target distance. The formula for obtaining the distance can be expressed as: (1)d=c2×t

Where d denotes the distance from the object to the LIDAR, c denotes the speed of light, and t denotes the time delay of the laser pulse, which is divided by 2 in the formula because of the round-trip distance traveled by the laser pulse.

Phase ranging is used to obtain distance information by measuring the phase difference generated by the round-trip flight of a continuous-wave laser signal between the radar and the target. To eliminate the effect of the phase difference on the distance measurement, the phase offset must be calculated first using the formula. The phase offset formula is: (2)Δφ = 2 π ×(ΔtT)

where Δφ denotes the phase offset, Δt denotes the time difference between the transmitted and received laser pulses, and T denotes the period of the laser pulse. Since the phase difference of a complete laser pulse is 2π, the phase offset can be calculated from the time difference.

Finally, the distance and phase offset formulas are combined to accurately find the distance between the object and the LIDAR, and the combination formula is expressed as: (3)d=(c2)×(t+Δφ2πf)

where f represents the frequency of the laser pulse. This formula takes into account the effect of phase difference on the distance, so it has higher measurement accuracy and stability.

Based on the relevant principles of LIDAR scanning, this paper chooses a 110KV substation as the research object and practical Ouster OS LIDAR to collect 3D point cloud data of the substation. The wide-angle 120° camera collects RGB images, and a total of 2048 data sizes are collected, which are named as BDZ dataset. It contains 1024 transformers, 740 fire sandboxes, and 5267 operators. The collected data can be used for tasks such as target recognition and 3D modeling of substations.

1)

Outlier denoising based on statistical filtering

Statistical filtering first needs to determine a neighborhood window size, which is usually determined by the characteristics of the point cloud and application requirements. For each point, statistical filtering will find all the points in its neighborhood window and calculate the mean and standard deviation values of the coordinates, distances, normal vectors and other parameters of these points. These statistical parameters can reflect the distribution characteristics of the point cloud in the local area and help to determine which points are outliers. Secondly, by comparing the statistical parameters of each point with the points in its neighborhood, it is judged whether the point is an outlier or not. Specifically, points with large differences from the mean or standard deviation of the points in the neighborhood are generally judged as outliers according to a set threshold or standard deviation. Finally, different methods can be selected to remove or replace the outlier points, and common removal methods include deleting the entire point or replacing it with the mean value of the points in the neighborhood. Statistical filtering outlier denoising is performed as follows:

Step1 Assume that there are n points in the point cloud data, for each point p_i(x_i, y_i, z_i), find its k nearest neighbor points p_j(x_j, y_j, z_j) in the 3D space by KD-Tree search algorithm.

Step2 Calculate the average distance d_i between the current point and its k nearest neighbor points. Then: (4)di=1k∑j=1k(xi−xj)2+(yi−yj)2+(zi−zj)2

Step3 Calculate the average of the global distances μ. Then: (5)μ=∑i=1n1ndi

Step4 Solve for the standard deviation σ. Then: (6)σ=1n∑i=1n(di−u)2

The key to outlier noise removal is to determine which points are outliers. After the processing of the above method, we need to determine whether d_i is within μ±aσ, and if not, then determine that it is an outlier point, so as to remove it. The parameter a and the number of point neighbors k need to be adapted according to the size of the data volume of the point cloud to be processed and the density of the point cloud. 2)

Voxel grid-based point cloud data downsampling

In order to speed up the point cloud target recognition, it is necessary to perform the downsampling operation first, i.e., to reduce the number of point clouds in a reasonable way, so as to improve the overall efficiency. In this paper, we choose to use the voxel grid method to downsample the substation 3D point cloud data. The specific operation steps are as follows:

Step1 Determine the side length of the voxel raster based on the set of coordinates of the point cloud data by taking the limit value X_max, X_min, Y_max, Y_min, Z_max, Z_min on each axis of X, Y, Z of the point cloud data. Then the side length of the largest enclosing box is: (7){ Lx=Xmax−Xmin Ly=Ymax−Ymin Zy=Zmax−Zmin

Step2 Divide the maximal inclusion box by voxel mini-grid. First set the side length of each voxellet grid as c, and then divide the X, Y, Z-coordinate axis where the point cloud data is located into M, N, L parts uniformly, the maximal containment box is divided into M×N×L voxellet grids. Where [-] denotes downward rounding and sum is the total number of voxel mini-grids. Then: (8)Sum=M×N×L (9){ M=⌊Lxc⌋ N=⌊Lyc⌋ L=⌊Lzc⌋

Step3 Assign the data points to the corresponding voxel rasters. Number each voxel raster as (i, j, k). The voxel raster number corresponding to the mnd data point (X_m, Y_m, Z_m) in the point cloud is: (10){ i=⌊Xm−Xminc⌋ j=⌊Ym−Yminc⌋ k=⌊Zzm−Zminc⌋

Step4 Simplify the point cloud. By calculating the center point coordinate (r_x, r_y, r_z) of each voxel mini-raster, it replaces all the point information in the original mini-raster. Combine the center points of all voxel rasters to realize the streamlined processing of the point cloud. The equation for solving the center of mass of voxel number (i, j, k) is: (11){ rx=1n∑m=1nPmx ry=1n∑m=1nPmy rz=1n∑m=1nPmz

In order to better realize the construction of substation 3D scene model, it is necessary to align the processed substation 3D point cloud data. In this paper, on the basis of the traditional ICP point cloud alignment algorithm, we optimize the ICP point cloud alignment algorithm by adding new point cloud descriptors with different weights to the matched point pairs with different degrees of importance, which can accelerate the convergence speed under specific convergence conditions [14].

The algorithm in this paper is improved on the basis of the traditional ICP algorithm, and the main improvement idea is to enhance the robustness of the point pair matching, when using KD-Tree for neighborhood search. The search condition of the nearest point is changed to the feature distance equation, i.e.: (12)min(fi(xi−xj)2+(yi−yj)2+(zi−zj)2+(ci−cj)2)

Also, feature distance filtering can be performed for matched point pairs based on this equation.

The normal-based evaluation function of the alignment error is chosen, i.e: (13)E(R,T)=argmin∑i=1i=N||(pi−(qjR+T))ni||22

where q_j is the point corresponding to p_i.

The specific implementation steps of the improved ICP point cloud alignment algorithm are as follows: 1)

Input the reference point cloud P and the point cloud Q to be aligned.

The SVD method is utilized to solve to obtain α, β, γ, t_x, t_y, t_z, which in turn yields the rigid transformation matrix [RT], which is rigidly transformed for Q. 8)

Calculate the evaluation function E_k(R, T) of the alignment error of this time, k representing the kth transformation. If the difference D_k = E_k − E_k+1 between the two times E_k before and after meets the set threshold then it is converged, otherwise go to step (4) and continue the iteration.

In this paper, the fusion between 3D point cloud and 2D color image is achieved by finding the correspondence between LIDAR point cloud coordinates and 2D image pixel coordinates in 3D space. At any moment, there is a monoclinic relationship between the 3D point cloud data and the image pixel coordinate system, and the mapping relationship between the two coordinate systems is expressed as: (15)s[ ui vi 1]=K[ R t 0T 1][ Xi Yi Zi 1],i=1,2,3...

Since the frame rates of the two sensors are not the same, the timestamps of the data acquired by the two sensors need to be aligned. In this paper, the Time Synchronizer in the ROS system is used to realize the alignment of the current frame of the 3D LiDAR point cloud data with the 2D image data in the color solid-state LiDAR scanning system.

Iterate the laser point in the point cloud frame, denote its coordinates as p(x, y, z), project the laser point under the 2D pixel coordinate system of the corresponding image through the mapping relation, and search for its nearest neighboring pixel point to obtain the RGB information, and then re-map it under the radar coordinate system [15]. At this time, the coordinates of the 3D color laser point are of the form: (16)pc=[x,y,z,R(u,v),G(u,v),B(u,v)]T

Where R(u, v), G(u, v) and B(u, v) denote the image color information of the projected pixel points, the point cloud (X, Y, Z, R, G, B) contains three-dimensional coordinates and color parameter information.

When the 3D laser point cloud is converted to the pixel coordinate system, the pixel coordinates are not integer, so the adjacent pixels in the 2D image are selected by calculating the Euclidean distance between the adjacent pixels and fusing the RGB values of the pixels to achieve the color fusion between the laser point cloud and the image pixels. Finally, the texture and color information in the 2D image are given in the 3D point cloud, so as to realize 2D-3D information fusion and provide effective data for the construction of 3D scene model of substation.

In order to realize the substation 3D scene model construction, this paper takes the Unity3D software and HTC Hive device as the basis, combined with the 3D point cloud data of the substation obtained from the previous collection, and obtains the substation 3D scene model construction process as shown in Figure 1.

Figure 1.

The 3D scene model construction process of the substation

Scanning the terrain data of the substation by LiDAR, using the acquired 3D point cloud data to select 3D Max 3D modeling tool to establish the 3D terrain model of the substation. Through the binocular stereo vision system to collect the image material of primary and secondary equipment of the substation, based on the established three-dimensional terrain model, use 3D Max software to establish the three-dimensional model of the substation’s fire fighting equipment and the scene based on the collected image material. The 3D model is imported into Unity3D software to develop the 3D scene of the substation, so that the scene has vividness and realism. The HTC Hive device is selected to realize the human-computer interaction of the 3D scene, which gives the substation operation environment a sense of scene and immersion. Utilizing the substation 3D scene for substation operation, the virtual simulation results are used to evaluate the safety of substation operation and provide guarantee for the safe operation of the substation.

YOLOv5 is a one-stage target recognition algorithm based on anchors, which can quickly identify the target and its relative position in the image. The YOLOv5 model consists of an input, a feature extraction module, a feature fusion module and an output, and each of them consists of several basic components, such as convolution, pooling, and activation functions [16]. 1)

Input module. YOLOv5s input module includes Mosaic data enhancement, adaptive computational anchor frame and adaptive scaling. Mosaic data enhancement addresses the problems of varying scale sizes and uneven distribution of data by randomly selecting images and reassembling them by scaling, splicing, and cropping, so as to make the model’s prediction of small targets more accurate. The adaptive computational anchor frame can be automatically adapted to the training set and validation set according to the set parameters. Adaptive image scaling is used to reduce the amount of computation by adjusting the aspect ratio of the image to a certain size, and then using the black edge strategy to unify the image size, thus speeding up the model training and inference.

Attention mechanism is an idea that mimics the human visual system and can enhance the deep learning model’s ability to focus on input information. Based on the neural network model of the attention mechanism, it can adaptively assign different weights according to the importance and relevance of the input data, thus helping the model to focus more on the useful information in order to better process the input data and improve the accuracy and generalization ability of the model [17].

Suppose there is an input sequence X = (x₁, x₂, …, x_n) and a target sequence Y = (y₁, y₂, …, y_m), where x_i and y_j denote the ith and jth elements in the input and target sequences, respectively. Then, given an input sequence X and a target sequence Y, the most common model of the attention mechanism can be represented as: (17)at=Attention(ht−1,X) (18)Ct=∑i=1nat,ihi (19)st=DecoderRNN([yt−1,Ct],st−1) (20)yt=OutputProjection(st)

The model is divided into two phases, i.e., encoding phase and decoding phase. In the encoding phase, the input sequence is encoded into a set of vectors H = [h₁, h₂, …, h_n]. where h_i denotes the encoded representation of the ird element of the input sequence. In the decoding phase, the target sequences Y are generated one by one, and the generation of each target sequence element y_τ depends on the previously generated elements as well as the coded representation of the input sequence.

In the above model, the role of the Attention mechanism is to provide contextual information for the decoding phase to better generate the target sequence elements. Specifically, s_t denotes the state representation of the first t elements in the target sequence, and Decoder RNN is a recurrent neural network used to compute the state representation in the target sequence. Attention is an attention computation function used to compute the correlation between the trd element in the target sequence and each element in the input sequence to obtain the attention vector a_τ. Each of the attention vector a_τ element represents the attention of the tth element in the target sequence to each element in the input sequence, which can be regarded as a set of weights. c_τ denotes a contextual representation of the input sequence obtained based on a_τ weighting, which is used to enhance the state representation s_t in the decoding stage, providing contextualized information related to the input sequence. y_t denotes the output obtained based on the tth element in the target sequence and c_τ.

The multi-scale feature fusion of YOLO architecture has experienced continuous optimization and improvement from FPN-like, FPN, and then PANet. From the principle point of view, the shallow network has higher resolution and covers more accurate location information. The deep network has a larger sensory field, covers more high-dimensional semantic information, and contributes more to the classification of the target. So optimizing the effect of information fusion at different scales becomes a way to enhance the network architecture.

The problem faced by single stage target detection algorithms is that it is not possible to get features of different scales by a single stage of feature extraction. The generated features extracted in the backbone segment are divided according to the stage, denoted as C₁, C₂, ⋯, C₇, and the number represents the number of times the image resolution is halved, e.g., C₄ indicates stage 4, and the output is a feature map of the original image of 1/16 size.

After that the feature fusion is performed layer by layer from top to bottom and the output is denoted as P. This process can be expressed as: (21)Pi=fi(Ci,Pi+1), i∈{3,4,⋯,6}

YOLOv5 applies PANet for feature fusion in the neck segment, and the PANet structure is characterized by a bottom-up fusion link established in layers C₃ to C₇, which enhances the upward transfer of strongly localized features from the bottom layer. Compared to PANet, BiFPN has 3 main improvements, firstly, some nodes are reduced, nodes with person degree 1 are deleted from BiFPN, because such nodes have no extra information compared to the previous node, which reduces redundant computation. Second, jump links are added so that the output layer not only gets the information that has been involved in feature fusion from the bottom up, but also retains the unfused information of the original node. Third, the fusion module is formed, which can continue to participate in heapden as a whole to do further fusion. The relationship of each layer is shown in the following equation: (22){ P7out=Conv(P7in) P6out=Conv(P6in+Resize(P7out)) P5out=Conv(P5in+Resize(P6out)) P4out=Conv(P4in+Resize(P5out)) P3out=Conv(P3in+Resize(P4out))

where Piin denotes the input of node P_i, Piout denotes the output of node P_i, and Resize(·) is the upsampling operation.

Through this design, a small increase in accuracy and a significant reduction in the amount of operations can be realized theoretically, which is of great practical significance for improving the target recognition and efficient monitoring of the 3D scene of the substation.

In recent years, the Attention Mechanism module has been widely used in computer vision tasks to tell the model what to pay more attention to and where, and is now used in deep neural networks in order to improve the performance of the model. However, in lightweight networks, the application of attention mechanisms is somewhat limited due to the fact that most of them incur additional computational overheads that are unaffordable for lightweight networks. Therefore, in this paper, we introduce a simple and flexible coordinate attention mechanism (CAM) with little additional computational overhead to improve the accuracy of the network.

Input features Fig. X is the output of the previous layer of convolution with dimension C × H × W, i.e., number of channels is C, height is H, and width is W. The average pooling using dimensions (H, 1) and (1, W) is used to encode the output of each channel along the horizontal vs. vertical coordinate directions, i.e., the c-th channel with height h vs. the cth channel with width w, respectively, with the following equation: (23)zch(h)=1W∑0≤i<Wxc(h,i) (24)zcw(w)=1H∑0≤i<Hxc(j,w)

The two transformations of the above equation perform feature aggregation along the two spatial directions and then cascade the two generated feature maps z^h, z^w and perform a convolution operation F₁ with a convolution kernel size of 1 to generate intermediate feature maps f for the pair of spatial information in the horizontal and vertical directions with the following formula: (25)f=δ(F1([zh,zw]))

Divide f into two single tensors f^h and f^w along the spatial dimension, and then transform the feature maps f^h and f^w into the same number of channels as the input X using two convolution operations F_h and F_w with convolution kernel size 1, as follows: (26)gh=σ(Fh(fh)) (27)gw=σ(Fw(fw))

In the formula, σ operation is the Sigmoid activation function, which is operated to get the range value from 0 to 1, which represents the degree of importance level. Expanding g^h and g^w as attention weights, the final output formula is as follows: (28)yc(i,j)=xc(i,j)×gch(i)×gcw(j)

The CAM is able to effectively attend to valid channels while focusing on spatial location coordinate information. In this paper, this attention mechanism is embedded into the CBL module in Backbone and the residual block in Bottleneck’s CSP to help the model to be more capable of feature extraction for the target of interest.

The loss function of YOLOv5 is at the heart of its training process and is responsible for measuring the discrepancy between model predictions and actual labels. In order to train the model to accurately predict the location, size, and class of the target, Improved YOLOv5 employs a composite loss function that consists of 3 main components, namely the bounding box regression loss, the object confidence loss, and the classification loss.

The object confidence loss L_conf is computed using a binary cross-entropy loss. This part of the loss is designed to optimize the model’s prediction of the presence or absence of the target, thus ensuring that the model accurately identifies the region containing the target. The formula is as follows: (29)Lossconf = −∑i=0K×K∑j=0MIijokj[C^ijlogCij+(1−C^ij)log(1−Cij)] −λmobj∑i=0K×K∑j=0MIijmobj[C^ijlogCij+(1−C^ij)log(1−Cij)]

Where K denotes that the final output feature map of the network is divided into K × K cells, M denotes the corresponding number of anchor frames in each cell, Iijaj denotes targeted anchor frames, Iijmobj denotes untargeted mismatched frames, and λ_noobj denotes the weight coefficients of the confidence loss for untargeted anchor frames.

The classification loss L_dass is calculated using the cross-entropy loss. This part of the loss ensures that the model can accurately classify each target detected. The formula is calculated as follows: (30)Lossclass=−∑i=0K×KIijobj∑[P^ijlogpij+(1−P^ij)log(1−pij)]

where pij is the predicted probability and P^ij is the actual probability of the category to which the target in the grid belongs.

In this paper, the CIoU loss is used to measure the difference between the predicted bounding box (including centroid coordinates, width, and height) and the real bounding box [18]. The CIoU loss improves on the IoU by taking into account the overlap between the bounding boxes, the centroid distances, and the aspect ratios to provide a more comprehensive assessment of the quality of the bounding box. The bounding box regression loss is calculated as follows: (31)Lciou=1−(IoU−ρ2(b−bgt)c2−αv)

Where IoU is the intersection and concurrency ratio of the predicted and real frames, b is the centroid of the predicted frame, b^g is the centroid of the real frame, and ρ is the Euclidean distance between the centroids of the predicted and real frames. c is the diagonal distance of the smallest rectangular region containing the predicted and real frames, α is the weight coefficient, and v is the difference between the aspect ratio of the predicted and real frames.

The total loss function of the improved YOLOv5 model is the weighted sum of the above three partial losses, i.e: (32)Loss=λ1Lciou+λ2Lconf+λ3Lclass

where λ₁, λ₂, λ₃ is the weighting coefficient used to balance the loss contributions of the different components. These weighting coefficients can be adapted to specific tasks and datasets to optimize the overall performance of the model.

In order to compare the effect of different downsampling methods in processing point clouds, this section carries out random downsampling, uniform downsampling and voxel downsampling on the original point cloud data of the substation respectively. By adjusting the size of the near-neighbor search radius in uniform downsampling and the resolution of the raster in voxel downsampling, the number of point clouds after uniform downsampling and voxel downsampling is ensured to be the same, which is 47251. The experimental data are shown in Table 1.

As can be seen from the table, the same from 281421 points downsampling to 47251 points, voxel downsampling time is the shortest, only 0.1049s, compared with the random downsampling and uniform downsampling were 17.66% and 31.53% faster, respectively. When voxel downsampling the original point cloud, for regions with high density such as the target object and its surrounding clutter, the probability of the points in it being sampled is greater, while for regions with low density such as the interference noise, the probability of the points in it being sampled is smaller. Instead, the voxel downsampled point cloud data filters out most of the interfering noise and also ensures important information about the target object and its surrounding clutter.

In order to compare the effects of different outlier removal methods and different parameters for processing the point cloud, this section denoises the downsampled point cloud using a statistical filter and a radius filter, respectively. When the statistical filter is used, the best results are obtained by adjusting the number of nearest neighbor search points k₁ to seek the optimal parameters. When the radius filter is used, the optimal parameters are sought by adjusting the size of the near-neighbor search radius r₁ and the size of the near-neighbor point set threshold k₂ to obtain the best results. The experimental data are shown in Table 2.

From the data in the table, it can be seen that when the radius filter is used for outlier removal, there is a linear relationship between the near-neighbor point set number threshold and the number of points after removing the outliers as well as the time taken, i.e., the larger the near-neighbor point set number threshold is, the fewer the number of points after removing the outliers and the longer the time taken. In the experiment with the best results, the number of point clouds is reduced from 47251 to 47103, and the elapsed time is 0.1225 s. When the statistical filter is used for outlier removal, there is no linear relationship between the number of nearest-neighbor search points and the number of points after removing the outliers, but there is a linear relationship with the elapsed time, i.e., the greater the number of nearest-neighbor search points, the longer the elapsed time. In the experiment with the best effect, the number of point clouds is reduced from 47,251 to 45,834, and the time used is 0.1517 s. Overall, this paper utilizes statistical filtering for the removal of outlier points from the substation 3D scene data with good effect, which can ensure the accurate processing of the substation 3D scene data.

In order to verify the feasibility of the improved ICP point cloud alignment algorithm proposed in this paper, the traditional ICP algorithm and the RANSAC+ICP algorithm are used to align the multi-view point cloud with 10 target equipment data in the 3D scene model of the substation, and the practical root-mean-square-error (RMSE) is used to measure the alignment accuracy. Table 3 shows the comparison results of RMSE and time consumption of different algorithms.

From the table, it can be seen that the traditional ICP algorithm is fast, but the alignment results are relatively poor. Although RANSAC+ICP improves the alignment accuracy compared with the traditional ICP algorithm, its alignment efficiency is not high, and its alignment accuracy is poor for some repetitive point clouds with relatively few repetitions. In comparison, the improved ICP point cloud alignment algorithm proposed in this paper can achieve more accurate and stable point cloud alignment results in various scenarios, although the alignment time is relatively long. In comparison, the average RMSE of the proposed alignment method is 93.73% higher than that of the traditional ICP alignment method, and 90.97% higher than that of the RANSAC+ICP alignment method. The method not only provides innovative improvements to the limitations of the traditional ICP algorithm, but also achieves significant results in improving the robustness and accuracy of the point cloud alignment. The improved ICP point cloud alignment algorithm proposed in this paper mainly uses KD-Tree to perform neighborhood search, changes the search conditions of the nearest points, and combines the SVD method to perform the perceptual transformation of the point cloud coordinates, which results in a significant improvement in the alignment effect of the substation 3D point cloud data. The algorithm is able to better describe the local geometric features of the substation 3D field point cloud, which improves the accuracy of the initial matching. The robustness of the algorithm to outliers and the accuracy of the initial estimation are further strengthened, making the overall alignment process more robust.

For the improved YOLOv5 model designed in this paper for substation 3D scene target recognition, the BDZ dataset obtained from the previous paper is divided into training set and test set according to the ratio of 8:2 as an example. The main evaluation indexes selected are accuracy (P), recall (R), average accuracy (AP), average precision mean (mAP@0.5 and mAP@0.5:0.95), recognition speed (FPS) and model volume (Vol). 1)

P-R curve of the model

The P-R curve of the model indicates the effectiveness of target recognition in the 3D scene of the substation, and the horizontal and vertical coordinates in its right-angle coordinate system are the recall rate and accuracy rate, respectively, and the AP value can be obtained by calculating the area of the lower part of the P-R curve in the coordinate system. The BDZ dataset constructed in this paper mainly contains three categories of transformers, fire sandboxes and operators, then its P-R curve is shown in Figure 3.

Figure 3.

The model of all kinds of target P-R curves

Based on the data in the figure, it can be seen that the AP values of the improved YOLOv5 model in this paper for recognizing the three categories of transformers, fire sandboxes and operators in the 3D scene of substations are 97.05%, 94.78% and 95.91%, respectively. The main reason for failing to realize complete recognition is that the complex spatial environment of the substation interferes with target recognition, and although this paper carries out certain preprocessing for the point cloud data, there are still certain defects in target recognition. Therefore, in the subsequent work, we will consider increasing the preprocessing of substation scene images, which will be used to improve the cleaning rate and recognition rate of images, and better guarantee the safety monitoring of substation operation and maintenance scenes.

In order to further validate the detection performance of the algorithm proposed in this study and to explore the effectiveness of each improvement method, ablation experiments were designed on the basis of YOLOv5, using the same hyper-parameters as well as training techniques for each group of experiments, and the results of the model’s ablation experiments are shown in Table 5.

As can be seen from the table, the introduction of the CAM module reduces the size of the model by 2.38MB, which is an effective lightweight method, and the mAP@0.5 is increased by 0.21 percentage points compared with the baseline model and 0.81 percentage points by mAP@0.5:0.95, although the network accuracy is not improved much, but it significantly reduces the number of parameters of the model. At the same time, after the introduction of BiFPN module to reduce the complexity of the model, although the model recognition speed is reduced to 103f/s, the indicators of mAP@0.5 and mAP@0.5:0.95 are increased by 0.09 and 0.14 percentage points, respectively. It is proved that the introduction of the BiFPN module improves the model’s ability to fuse the multi-scale features of the target in the 3D scene of the substation, and makes the identified target frame more suitable for the contour of the object. Compared with YOLOv5, the overall recognition speed of the improved YOLOv5 model is reduced by 36.95%, while the mAP@0.5 and mAP@0.5:0.95 are increased by 2.25 percentage points and 5.02 percentage points, respectively. The model has further strengthened the ability to identify and fit the 3D scene target of the substation, and the hardware requirements are smaller during the network operation, which can be widely used in the automatic task of the 3D scene target of the substation with high requirements for the IOU of the target box and the need for more accurate positioning.

In order to verify the effectiveness of the improvement modules on the baseline method, the cross-sectional comparison experiments of each improvement module are now conducted based on the YOLOv5 version. In the cross-sectional comparison of attentional mechanisms, three typical attentional mechanisms, SE, BAM, and CBAM, are selected as the control group for performance comparison. In the experiments, it is guaranteed that different attention mechanism modules are introduced at the same position in the backbone network and the neck feature fusion network, and at the same time, it is guaranteed that other network structures remain unchanged. In order to verify the effectiveness of BiPAN neck network, four typical network structures, FPN, PANet, GFPN and PRFPN, are selected as control groups for comparison. In addition, five traditional loss functions, IoU, GIoU, DIoU, EIoU and SIoU, are selected for the cross-sectional comparison experiments. Table 6 shows the results of the cross-sectional comparison experiments of the improved modules.

Based on the data in the table, the following conclusions can be drawn: 1)

Compared with the introduction of SE, BAM and CBAM and the absence of attention mechanism, the mAP@0.5 increased by 1.39 percentage points, 1.15 percentage points, 0.66 percentage points and 2.24 percentage points, respectively, which verified the effectiveness of CAM attention mechanism in improving the YOLOv5 model.