Machine Learning Based Outlier Detection Algorithm for Distributed Flexible Sensing Module with Non-stationary Multi-Parametric Data

Advanced Measurement System (AMI) an important technical support for smart grid power metering, is a complete network system used to collect, store, analyze and apply user information on electricity consumption, consisting of home EMS, power meters, local communication network, remote communication network, metering data management system and data integration platform [31]. Figure 1 shows the schematic structure of AMI, which is a complete metering system mainly composed of several key technologies and applications that can record the user’s electricity consumption behavior in near real time and transmit the collected data to the metering data management system (MDMS) through the network.

Figure 1.

Diagram of AMI structure

1)

The MDMS is a database with analytical tools capable of processing and storing metered values from energy meters, and one of its basic functions is to validate the legitimacy, edit and evaluate the data of the Advanced Measurement System (AMS) to ensure the completeness and accuracy of the messages. Through the aggregation, analysis and storage of messages, a reasonable tariff strategy is formulated to ensure that when the communication network is interrupted and the user side fails, the grid is still able to accurately demand forecast without affecting the stability of the entire system.

In terms of communication mode, the uplink communication of flexible sensing terminal device supports wired (Ethernet, 485, HPLC, etc.) and wireless (infrared, Bluetooth, etc.) communication modes, and the energy controller and the cloud master prioritize wireless public network communication, and 5G or wireless special network communication can be used in accordance with the network development. Downstream communication supports HPLC, Bluetooth and other communication modes, and HPLC communication is the main communication mode between gateways, with a broadband of 2-12MHz, a rate of 1Mbps or more, and an average online rate of 99.95% or more, which meets the needs of massive low-voltage equipment access and real-time data interaction, and different communication modules can be selected according to different environments. Distributed flexible sensing terminal application system architecture mainly connects the distribution side, branch side and meter box side to collect the metering data of the power meter in real time, supplemented by realizing the effective analysis of the power meter data.

AMI-based online monitoring of power metering device is mainly to realize the management of power metering device operation status, through the information collection of power metering device operation status, data analysis and processing, and logical judgment to the online monitoring of the basic operating parameters of the power metering device. Specific executive functions include online power collection and monitoring and power error testing unit, current CT switching unit, voltage PT switching unit and power meter pulse collection and switching unit, of which the power collection and monitoring and power error testing unit is mainly for the current circuit of the power meter error, the secondary load of the transformer and the PT secondary voltage drop of the online monitoring. The current CT switching unit is responsible for the secondary switching of the current loop, the voltage PT switching unit is responsible for the switching of the multi-section PT bus, and the power meter pulse acquisition switching unit is responsible for the switching of the power pulse input of the measured power meter [32].

The basic composition of AMI-based online monitoring of power metering device mainly includes device abnormality knowledge base, cloud platform storage unit, a new generation of Internet application technology, interface module and background management, etc., which is capable of accomplishing data interaction between multiple subsystems of power metering device. Online monitoring is based on the power collection terminal, and the main function of the collection terminal is to collect, read and store the data signals of the monitoring points of each power measuring device. After obtaining various types of data through the collection terminal, the collected data are analyzed online, which can provide an effective and reliable basis for the fault diagnosis process.

By designing a modular measurement equipment structure, we have innovated from a traditional decentralized structure to a “building block” modular structure. In addition to ordinary measurement and sensing functions, the innovative design of the “expansion module” function. According to the actual demand, through the “hot-swappable interface” to access the powerful special expansion module, to realize the directional expansion function. In order to ensure that the connection between the modules is reliable, the design of the module connectors using a single row of pin-type interfaces, the use of alloy gold-plated plug design, and in the upper and lower ends of the module to increase the position of the fixed buckle, to ensure that the module splicing operation is stable, vibration, shock resistance, expansion module design as shown in Figure 2.

Figure 2.

Extension module design drawing

At present, according to the application requirements of different work scenarios, the expansion module can be divided into intelligent protection module, displacement control module, environmental awareness module and so on. Module can be extended to install the protection unit door lock built-in opening and closing control device and camera device, the module built-in all the way to the magnetic sensor connected to the door lock, while the built-in camera device. When there is illegal intrusion, the door lock within the opening and closing control device through the magnetic sensing signals to inform, trigger the two-way camera to take continuous pictures, stored locally, and through the upload master station, triggering early warning. At the same time, the built-in strong magnetic monitoring module can monitor the magnetic intensity around the box in real time, and report to the main station in time when it exceeds the threshold, triggering an early warning.

Depending on W = P · t, W = U · I · t or W = U² · t/R = I² · R · t, 1 degree is 1kW·h equal to 3.610⁶ joules. Among them, the significance of electric energy and power is different, but its mathematical expression is only expressed in the time parameter, the electric power field research on electric energy measurement is mainly based on the measurement of electric power, through the meter to complete the quantity conversion between electric power and electric energy, in the expression of electric energy can be expressed in terms of electric power. Two-part tariffs are widely implemented in China, mainly based on active power as the basis for the collection of electricity, reactive power measurement is mainly used in the assessment of the user’s power factor. General small-scale residential power measurement analysis are mainly active measurement principle, can analyze the impact of power meter measurement error. Usually include voltage and current sampling error, measurement chip and register parameters configuration error, measurement chip and register, CPU operation processing error, the error of the above data directly determines the power measurement error. Practice shows that only the comprehensive error of the power measurement device is to measure the accuracy of the power measurement device is the only indicator, so by calibrating the above problems one by one, you can basically complete the power meter collection error calibration. Specific calibration steps are as follows: 1)

Read data such as energy meter clock, battery undervoltage status, programmed open cover event, peak and trough correspondence with total power, etc., to detect the initial state of operation and analyze whether there is any problem with the energy meter system.

For the multi-parameter electric energy metering data in the non-stationary environment, it obtains the relevant temperature and humidity data through the protection unit of the distributed flexible sensing terminal application system, but due to the high-dimensional characteristics of the data it collects, it is not conducive to outlier detection in the later section. Based on this, in order to improve the efficiency of the original variables with smaller variables in a linear combination of most of the data, this paper introduces the principal component analysis (PCA) to carry out the dimensionality reduction of the original data of the electric energy multi-parameter. PCA is a multivariate statistical method, through the dimensionality reduction with a few principal components to express the original multiple variables in a linear combination of the original way [33].

Let x=(x1,x2,…,xp)′ be a p-dimensional aggregate and consider the following linear variation: (1)y1=a11x1+a12x2+…+a1pxp y2=a21x1+a22x2+…+a2pxp ⋮ yp=ap1x1+ap2x2+…+appxp}

Let x₁,x₂,…,x_n be a sample taken from aggregate X, where xi=(xi1,xi2,…,xip)′(i=1,2,…,n) . The matrix of sample observations is: (2)X=[ x11 x12 … x1p x21 x22 … x2p ⋮ ⋮ ⋮ xn1 xn2 … xnp]

X Each row corresponds to one sample and each column corresponds to one variable, the sample covariance matrix and the sample correlation coefficient matrix are, respectively: (3)S=1n−1∑i=1n(xi−x¯)(xi−x¯)′=(sij) (4)R=(rij),rij=sijsiisij

where x¯=1n∑i=1nxi is the sample mean. S serves as the estimate of ∑ and R serves as the estimate of R the sample principal components from S or R.

The contribution rate indicates the ability of the principal components to synthesize the original variable, which is defined λi/∑j=1pλi(i=1,2,…,p) as the proportion of the variance accounted for by the ith principal y_i component, with the rule being that the contribution rate of the principal components decreases gradually. The contribution ratio of the first m principal components and ∑i=1mλi/∑j=1pλj is the cumulative contribution ratio of the first m principal components.

The principal elements in PCA are the information produced by several variables of the data to be processed after a linear combination, and this change makes the variance of the variables of the data to be processed after the transformation is performed.

After the completion of data dimensionality reduction, based on the maximum likelihood method to calculate the separability of various types of anomalies in the non-stationary environment of multi-parameter power measurement data, when the higher the separability obtained by the calculation, it means that the data item has a more obvious classification effect for the anomalies, and select the data item with higher separability as the feature data item of various anomalies. The calculation process of feature data screening based on maximum likelihood method is as follows: 1)

Data preparation. Preliminary screening of smart energy meter metering data, by calculating the Pearson correlation coefficient between each feature and the target anomaly, selecting features with correlation coefficients close to 1 or -1, and filtering out features with higher correlation with the anomaly as alternative features.

Where ρ(y|normal) refers to the probability density of the feature data appearing in normal sample data, normal denotes normal sample data, y denotes the feature data, ρ(y|abnormal) refers to the probability density of the feature appearing in abnormal sample data, and abnormal denotes abnormal sample data. 3)

Calculate separability. Calculate the separability of various types of abnormalities caused by different alternative feature data according to the maximum likelihood value. The higher the separability, the more obvious the classification effect of the feature data on the anomalies.

The formula is as follows: (6)separability=2×L(y)×(J1+J2)

Where separability denotes separability, J₁ refers to the separability information of the anomaly class for the non-anomaly class, and J₂ refers to the separability information of the non-anomaly class for the anomaly class.

The formula for J₁ is as follows: (7)J1=R[L(y)]=∫ρ(y|x1)lnρ(y|x1)ρ(y|x2)dy

where R is the KL scattering function, ρ(y|x1) refers to the maximum similarity of the anomaly class, x₁ denotes the normal sample data, ρ(y|x2) refers to the maximum similarity of the non-anomaly class; and x₂ denotes the anomaly sample data. The formula for J₂ is as follows: (8)J2=R[1L(y)]=∫ρ(y|x2)Inρ(y|x2)ρ(y|x1)dy

4)

Screening feature data. According to the value of separability, the feature data items with higher separability are screened out, and these feature data items can be used as the basis for building the feature model. Specifically a threshold needs to be set, if the separability of a feature data is higher than this threshold, it is considered to be a valid feature data. For the screened valid feature data, it will be used as the input of the precision research function.

The main task of electric energy metering data point clustering is to aggregate metering data objects with similar characteristics into distinct clusters. In this paper, the fitting method based on the clustering results of electric energy metering data points does not allow specific prediction of the final clustering results of the electric energy metering dataset before the clustering starts. The number of class clusters formed in the clustering result is unknown, so algorithms that require the number of class clusters in the clustering result as an input condition are no longer applicable when selecting a clustering algorithm. Among the machine learning methods, the density-based DBSCAN clustering algorithm satisfies the above constraints, so the electricity metering data point clustering method in this paper is improved on this basis [34].

Assuming a data set of D, D={x1,x2,⋯,xn} and an object threshold parameter of Mpts, the concepts and definitions related to the DBSCAN algorithm are as follows: 1)

R Neighborhood set, NR(xj) For a data object x_i ∈ D, its R neighborhood set is the set of sub-data objects in the data set D whose distance from x_i is not greater than R, i.e., NR(xj)={xi|d(xi,xj)≤R} and the number of sub-data object sets is |NR(xi)| .

The Local Outlier Detection (LOF) algorithm aims to find outliers in a dataset, where the data points are outliers or not depending on the local environment. The basic principle is to quantify the density of data within the neighborhood of the data point and then achieve outlier detection by the degree of denseness. First calculate the local reachable density of the detection point, and then calculate the outlier factor of the detection point, which can characterize the data density around it. The larger the value of the outlier factor, the higher the outlier degree of the data point i.e., the higher the possibility of data anomalies, and the smaller the value of the outlier factor, the lower the degree of the outlier degree, and the lower the possibility of data anomalies [35]. The process of calculating the LOF is as follows:

Construct the sample matrix from the sample data extracted in the previous section based on PCA and DBSCAN clustering, and construct the sample matrix O ∈ R^m×n, i.e.: (9)O=[O1,O2,⋯On]=[Q1,Q2,⋯,Qn] (10)Qi=[Qi(1),Qi(2),⋯Qi(m)]

where m denotes the number of sampling points. Waiting for the anomaly judgment is the Q statistic q = Q_i(m+1) at the m+1nd sample point, and the Euclidean distance between data point q and data point o ∈ O_i is d(q,o). First, the Euclidean distances d=[di1,⋯,dik,⋯dim] , (d1>,⋯,dk>,⋯dm) of all the data points in data points q to O_i ∈ R^m×1 are computed and sorted in descending order, and the kth-order distance d_k−distance of data point q is: (11)dk−distance=dk

The set of all data in dataset O_i that satisfy this condition of d(q,o) ≤ d_k−distance form the kth order neighborhood N_k−distance of datapoint q. The reachable distance d_kr(q,o) from datapoint q to datapoint o is: (12)dkr(q,o)=max{d(q,o),dk−distance}

Simply put, a small distance between data points is d_kr(q,o) between data points is d_k−distance between two data points.

The local reachable density lrd(q) of a data point q is the reciprocal of the average d_kr(q,o) of all data points within it N_k−distance, calculated as: (13)lrd(q)=(∑o∈Nk−distance(q)dkr(q,0)N(q))−1

Where N(q) represents the number of other data points in the k-order neighborhood N_k−distance of data point q, and the smaller the value of lrd(q), the greater the probability that data point q is an outlier.

The local outlier factor lof(q) of data point q represents the ratio of itself to the mean value of lrd(o) and o ∈ N_k−distance(q) of other data points in N_k−distance. Then: (14)lof(q)=∑o∈Nk=distance(q)lrd(o)lrd(q)N(q)

If data point q deviates from the overall O of the data, i.e., lrd(q) is below the normal level within its neighborhood, this can lead to a disproportionate surge in the local outlier factor lof(q) for data point q. This is manifested by the fact that if data point q is not an outlier, it lof(q) approaches 1. If data point q is an outlier, it lof(q) ≫ 1, which indicates that there are error-varying individuals in the monitored population.

Since the Q-statistic also couples the error change dynamics of other power metering data, it is necessary to locate the power metering data where the error change occurs after the error change is detected. According to the theoretical derivation, compared to the local outliers in the significant case, the possibility of error drift increases in the case where the latest calculated local outliers of the energy metering data are shifted the most. Therefore, by quantitatively analyzing the relative displacement of the local outliers of each power metering data and arranging them in order, it is possible to accurately pinpoint the power meters that are experiencing error drift. In addition, in order to more accurately describe the relative displacement of the local outliers of each meter, we introduce the concept of contribution, and the meter with the largest contribution can be localized as the meter with the error drift. Its calculation expression is: (15)Gi−1=lofcilofi⋅∑j=1nlofjlofcj

where G_i, lofi(lofj) , and lofci(lofcj) denote the relative displacement contribution of the ith (jth) CVT, the local outlier factor of the m + 1th sampling point, and the local outlier factor under significant conditions, respectively. lof_c can be obtained by superimposing the maximum error value of the corresponding accuracy class on some of the continuous data in the first m sampling points of the corresponding energy meter, and then calculating the superimposed Q statistics with the local outlier factor.

Since the number of outliers in multicomponent power metering data in non-stationary environments is small as a proportion of the overall dataset, in order to make outlier mining more targeted, the overall dataset is first preliminarily screened by clustering. Density-based DBSCAN clustering does not require pre-specification of the number of clusters to be clustered and is able to find any number and shape of clusters in the noisy dataset containing noise, but the algorithm requires the user to set the parameters ε and Minpts without prior knowledge, and the parameters have a great impact on the quality of the final clustering results. In order to overcome the sensitivity of the input parameters, input K-nearest neighbors are used instead of parameters ε and Minpts. When judging the outliers of the screened multi-parameter electric energy metering data in a non-stationary environment, due to the complexity and large computational volume of the LOF calculation, for this reason, a new local outlier factor LAOF is constructed to measure the anomaly degree of the screened data. 1)

Processing of DBSCAN input parameters

In order to improve the clustering quality and reduce the resource consumption, the idea of K-nearest neighbor is introduced into the DBSCAN clustering algorithm to obtain the distribution of the data set while calculating the k-distance, and the a priori knowledge of the data set is obtained by analyzing the input k-neighbor number instead of the density threshold Minpts. And a certain number of data objects k − dist(p) is taken to calculate the mean k − mean instead of the spatial radius ε, which reduces the effect of the multi-parameter inputs on the clustering results and The core points are newly defined according to the improved method, and the number of data objects is selected based on the a priori knowledge acquired from the dataset.

where the reachable distance formula is: (17)reachk−dist(p←o′)=max{k−dist(p),dist(p,o′)}

Since the LOF algorithm calculates the reachable distance and reachable density with a high time complexity of o(Nk2) when dealing with larger datasets, the relevant definition of the algorithm is improved appropriately by using the K distance of the data object as the reachable distance and using it as the radius of the region to obtain the area of the region instead of the distance sum, with a time complexity of o(Nk), and constructing a new local outlying factor LAOF with a new definition of it.

The local density of object p can be expressed as: (18)LA(p)=|Nk(p)|π*(k−dist(p))2

The local density of object p is calculated as the number of data points per unit area within the circle area centered on p and radiused on k − dist(p), which greatly reduces the time complexity of the calculation compared to the calculation of the local reachable density of data objects in the LOF algorithm.

According to the formula for solving the local outlier in the LOF algorithm, the formula for calculating the local outlier in the LAOF algorithm is analogous.

The local outlier factor of object p can be expressed as: (19)LAOF(p)=∑o∈Nk(p)LA(o)LA(p)*|Nk(p)|

The local outlier factor of object p is expressed as the inverse of the ratio of the local density mean of LA(p) to that of its k neighbors. Obviously the smaller the local density of p, the larger the local density mean of k neighbors of p, and the larger the local outlier factor of p, the higher the degree of outlier of data object p.

In this section, three low-voltage stations (A, B, and C) in an area of C city, Province H, are analyzed and validated as an example, and data dimensionality reduction and feature filtering are performed using historical power metering data collected by State Grid H Province Electric Power Co. through the collection system of distributed flexible sensing terminals. Figure 3 shows the feature extraction results of the power metering data, in which Figures 3(a)~(d) are the voltage peak-to-peak, craggyness, skewness, and waveform factor features, respectively.

Figure 3.

Electrical energy measurement data characteristics

As can be seen from the figure, the power metering data of different stations show obvious differences after feature extraction. Observing the voltage peak-to-peak characteristics, the peak-to-peak values of power metering of A, B and C stations fluctuate around 10, 6 and 30, respectively. From the voltage crag characteristics, the voltage crags of A, B and C types of stations fluctuate around 2.5, 1 and 3, respectively. From the voltage skewness characteristics, the voltage skewness of A, B and C type stations fluctuates around 1.25, 0.5 and -1, respectively. From the characteristics of voltage waveform factor, the waveform factor of A and B type of stations fluctuates around 1.005, while that of C type of stations fluctuates around 1.05. Considering the above features comprehensively, the feature extraction effect of power metering data from different stations is more satisfactory, and the feature differentiation between different stations is more obvious, which lays the foundation for the subsequent feature downscaling and clustering analysis.

Since there may be a certain degree of correlation between features of electricity metering data, the purpose of dimensionality reduction is to improve the data processing speed by removing noise and unimportant features while retaining the most important features. Since the PCA method can effectively maintain the features of high-dimensional data and is also suitable for scenarios in which high-dimensional data are downscaled to low-dimensional data and visualized, it is used to demonstrate the visualization results of the recognition method in this study. In order to facilitate the calculation, the PCA algorithm is applied to downsize the power metering feature dataset, and the original power metering dataset collected is T*n dimensional, and a 6*n dimensional power metering feature dataset is formed by power metering feature extraction. Further based on the PCA algorithm, the power measurement feature data is downscaled to 2*n-dimensional low-dimensional feature data. The clustering of data after dimensionality reduction is shown in Fig. 4, and Fig. 4(a)~(d) shows the feature dimensionality reduction by selecting the cosine distance, Chebyshev distance, Euclidean distance, and Mahalanobis distance metrics, respectively.

Figure 4.

Degraded data clustering

As can be seen from the figure, by comparing the clustering under different clustering distances, it can be intuitively seen that it is relatively ineffective when using the Mahalanobis distance metric for feature dimensionality reduction. However, the cosine distance, Chebyshev distance, and Euclidean distance are all effective in differentiating the power metering feature data of a station. Since the PCA algorithm cannot maintain the original physical meaning of the original dataset after mapping the high-dimensional data to the low-dimensional space, the distances between different clusters in the visualization results become meaningless and cannot represent the similarity. Considering the advantage of Euclidean distance in computational speed, this paper chooses Euclidean distance for PCA dimensionality reduction.

In this paper, the improved DBSCAN clustering algorithm is used for data feature clustering when performing outlier detection of multi-parameter power metering data in a non-stationary environment. For the effectiveness of the improved DBSCAN clustering algorithm, this paper uses the contour coefficient to measure the clustering quality of the algorithm. The numerator of the contour coefficient (CC) is the measure of the “empty space” between two clusters, and the denominator is the larger of the two lengths, i.e., the radius of the cluster and the distance between the two clusters. The contour coefficient value ranges from -1 to 1. A negative value of the contour coefficient indicates that the radius of the cluster is larger than the distance between the two clusters, indicating that the clusters are overlapping. A larger value of contour coefficient indicates a higher quality of clustering in the clustering algorithm. Therefore, the average value of the contour coefficients of all points within a single cluster is used to measure the clustering quality of the whole cluster as a measure of the clustering effect of the improved DBSCAN clustering algorithm. Based on the power metering data of three low station areas obtained in the previous section, 2000 data were randomly selected from them, which contained seven categories. Hybrid Gaussian model (GMM), load essential features (COL), and matching analysis method (MAA) are selected as comparison algorithms in the experimental process, and experiments are conducted by ten-fold cross-validation method, and Table 1 shows the clustering effect of different clustering algorithms on electric energy metering data.

In order to further verify the effectiveness of the improved DBSCAN clustering algorithm in this paper, the computation time of different clustering algorithms is compared and analyzed by selecting the hybrid Gaussian model (GMM), the load essential features (COL), and the matching analysis method (MAA) as the comparison algorithms, and the results of the comparison are shown in Fig. 5.

Figure 5.

Calculation time comparison results

As can be seen from the table, the mean value of profile coefficient obtained by ten-fold cross-validation of the improved DBSCAN algorithm for feature clustering of multi-parameter electricity metering data is 0.705, which is 81.23% higher compared to the mean value of profile coefficient of the original DBSCAN clustering algorithm. Compared with MAA, COL and GMM algorithms, the mean contour coefficient of the improved DBSCAN clustering algorithm proposed in this paper is improved by 59.14%, 57.37% and 39.05%, respectively. Therefore, it indicates that the improved DBSCAN clustering algorithm in this paper obtains better clustering results when performing feature clustering of multi-parameter power metering data, and can provide reliable clustering results for realizing outlier detection of multi-parameter power metering data in non-stationary environments.

According to the comparison results of different algorithms in Fig. 5, the computation time of non-stationary multi-parameter power metering data clustering based on the improved DBSCAN clustering algorithm shows a relatively smooth state, and when the number of iterations reaches 80 times, its computation time is 3.33 min, and the mean value of the computation time for the 80 times of iterations is 3.38 min. while the computation time of the 80 times of iterations of the non-stationary multi-parameter power metering data clustering based on the hybrid Gaussian model, the load essential features, and the matching analysis algorithm is 3.33 min. analysis algorithms, the average computation time of non-stationary multi-parameter energy metering data clustering is 4.45, 5.13 and 6.32 min, respectively. The average computation time of this paper’s improved DBSCAN clustering algorithm in clustering is reduced by 24.05%, 34.11%, and 46.52% compared to the three comparative algorithms, and the computation time of this paper’s improved DBSCAN clustering algorithm is lower than that of the traditional The computing time of the improved DBSCAN clustering algorithm in this paper is shorter than that of the traditional clustering algorithm for electricity metering data. This fully reflects the effectiveness of this paper’s improved DBSCAN clustering algorithm in non-stationary multi-parameter energy metering data clustering, and the algorithm has a faster clustering time and better performance in energy metering data calculation.

For the non-stationary multi-parametric data outlier detection algorithm proposed in this paper, this paper tests the algorithm on the basis of simulated data set real data set. Laboratory, the algorithm of this paper is compared with LOF, COF, LDOF algorithms to analyze the effect of the time efficiency of this paper’s algorithm, the accuracy of the outlier detection of the outlier detection of the combined parameters of the algorithm on the accuracy of the algorithm detection.

In this paper, the algorithm is first validated on a simulated dataset, the data distribution in the simulated dataset is sparse but shows a certain pattern, the dataset contains 24 data, mainly consists of two diagonal lines with an angle of 45° with the x-axis, each diagonal upward with 12 data points, and its specific distribution is shown in Figure 6. The two oblique lines are compared to the coordinate origin a, b is a point on the x-axis, and c is the farthest point from the origin on the oblique lines. Intuitively, a and c are the data that show a certain pattern distribution, and b is the data whose displacement deviates from the pattern distribution, so the point with the highest degree of sharpness should be point b.

Figure 6.

Simulated data set

On the basis of the simulated dataset, the accuracy of this paper’s algorithm in detecting outliers and the effect of parameters on the algorithm are verified. According to the experience take the parameter k equal to 6, the experimental results are shown in Fig. 7, the x-axis represents the data sequence in the simulated dataset, and the y-axis represents the values of different outlier factors. By analyzing the data points with the largest outlier factor for each algorithm it is known that it is the COF algorithm and the algorithm in this paper that can detect the outlier data b. It is proved that in the face of sparse data distribution and presenting a certain pattern distribution, this paper’s algorithm inherits the advantages of LOF and can accurately detect the outlier data outside the pattern.

Figure 7.

Comparison of detection results with the same parameters

Secondly, this paper further designs the simulation dataset, which mainly consists of three data classes X1~X3 with different densities and five outlier data consisting of Y1~Y5, and runs four different algorithms on this dataset. In order to facilitate the analysis, only the detection accuracy of the algorithms under different parameters, i.e., the number of correctly found forest clusters/total number of outliers, is compared. Figure 8 shows the comparison results of the different algorithms on the simulated dataset.

Figure 8.

Changes of accuracy rate with different parameters

As can be seen from the figure, most of the algorithms can detect the outlier data more accurately when the parameter k is set reasonably, but with the increase of the parameter k, the accuracy of LOF algorithm and COF algorithm shows a significant decrease, while the accuracy of LDOF algorithm and the algorithm of this paper is almost unaffected by the parameter k, and their accuracy of the detection of the outlier points is 59.42% and 100%, respectively. Because the setting of parameter k directly affects the calculation of the kth distance field of the data object, when k is set unreasonably, the value of the outlier factor of the outlier points close to the high-density data region will be reduced, or even lower than the value of the outlier factor of the data points at the edge of the high-density region, thus affecting the detection results of the LOF algorithm and the COF algorithm. Compared with the algorithm in this paper, although the LDOF algorithm avoids the influence of the parameters on the detection results by calculating the ratio of the k-nearest-neighbor distance of the data object to the k-nearest-neighbor internal distance, the k-nearest-neighbor distance of the edge data is larger while the k-nearest-neighbor internal distance is smaller, resulting in a generally larger value of the outlier factor of the edge data, which affects the accuracy of the detection of the outlier points of the data.

In order to further verify the effectiveness of this paper’s algorithm on high-dimensional data with more data volume, based on the electricity metering data of three low stations selected in the previous paper, five categories of data points from them are selected as the cluster data, and then eight categorized data points are taken as the outlier data. Use the improved DBSCAN algorithm to perform clustering first, and compare the outlier detection accuracy and running time of different algorithms on the real dataset. Fig. 9 shows the outlier detection effect of different algorithms, where Fig. 9(a)~(b) shows the outlier detection accuracy and running time, respectively.

Figure 9.

The effect of the survey of the group

As can be seen from the figure, for different values of parameter k, the accuracy of this paper’s algorithm is higher than that of the other three algorithms. The accuracy of this paper’s algorithm is up to 0.845 for outlier detection of non-stationary multi-parameter power metering data, while the LOF algorithm, COF algorithm, and LDOF algorithm have outlier detection accuracies of up to 0.667, 0.575, and 0.727 only. In addition, the running time of LOF and COF algorithms grows rapidly with the increase in the amount of data, and the running time of the LDOF algorithm is lower than that of the COF algorithm. However, the running time of the algorithms in this paper grows with the amount of data, and its running time is only 345.07ms at the highest, which is shorter than that of the other three algorithms. It shows that the advantages of the outlier detection algorithm designed in this paper will be more prominent when the data volume is large. Therefore, the algorithm in this paper has better performance when performing outlier detection for non-stationary multi-parameter power metering data.

The article obtains power measurement data in distributed flexible sensing terminal, in which the measurement data are collected through the protection unit, and the protection unit will be affected by the temperature to a certain extent, which makes the acquired data have a large error. In order to analyze the trend of the electric energy measurement data of different systems at different temperatures, this paper selects the data acquisition system based on big data technology and cloud computing technology as a comparison, so as to verify the effectiveness of the system in this paper in the acquisition of electric energy measurement data under temperature change. Figure 10 shows the trend of the ratio difference of electric energy metering data under different temperatures.

Figure 10.

Error comparison diagram under different temperatures

After analyzing, it is found that at low temperature (-25℃~0℃), the range of variation of the ratio of electric energy metering data of this paper’s system affected by temperature is between -0.5%~-0.3%, and the range of variation of the ratio of electric energy metering data of the Big Data and Cloud Computing systems is [-0.1%,0.05%] and [-0.1%,0.1%], respectively. In the case of room temperature (0℃~18℃) and high temperature (18℃~25℃), the range of variation of the specific difference of electric energy metering data of the system in this paper affected by temperature is between -0.35% and -0.2%, and the range of variation of the specific difference of the other two systems fluctuates more. By analyzing the results, for different types of power metering data acquisition system, temperature has a significant impact on the change of power metering data error. In the low-temperature state than the difference change is small, room temperature state than the difference change increases, in the high-temperature state than the difference change is large, than the difference change is positively correlated characteristics. Comparatively speaking, the specific difference of the power metering data obtained by the system in this paper is relatively small, which can provide reliable data support for the accurate realization of outlier detection, and also shows the effectiveness of the application of the distributed flexible sensing terminal in the non-steady state environment.

The outlier detection of distributed flexible sensing terminal data is carried out to better realize power dispatching, based on which, combined with the distributed flexible sensing terminal application system designed in the previous section, it is applied to the error analysis of monthly power total active power in the distribution network. Based on the total active power error data from 2023-05 to 2024-04 in City C, Province H, the error of the analog output of the system is compared with the actual output. Figure 11 shows the monthly electrical energy error of the electrical energy metering system.

Figure 11.

The monthly electrical error of the electric power measurement system

From the figure, we can get that the maximum actual monthly total active power error is -0.471% (2023-07), and the variation of monthly power error in 1 year time is 0.227% (2023-09, 2023-07 difference), and the overall metering performance is stable during one year operation, which can be prioritized to be applied in electric energy trade settlement. Distributed flexible sensing terminal application system due to which the B-phase meter failure, the maximum total active monthly power error is only -4.435% (2024-01), for the normal operation of the A and C phases, the maximum monthly active power error is -0.272% (2023-11) and -0.199% (2023-07), but its 1-year power error variance is only 0.212% and 0.289%, distributed flexible sensing terminal application system of electric energy metering data collection error is small, the normal operation of the metering equipment also exists in the case of large system error, can be put into operation through the parameter configuration to eliminate the system error and then used for electric energy metering. In summary, the distributed flexible perception terminal application system designed in this paper can realize the accurate acquisition of power measurement data and meet the demand for power distribution network power allocation.