Deep Learning Models in Network Intrusion Detection Systems

ResNet-CNN [31] is a model for deep learning that was originally designed to address the problem of gradient vanishing or gradient explosion, which is common during deep neural network training. Its main feature is the introduction of residual blocks that comprise multiple convolutional layers and specially designed residual connections. These residual connections enable the input information to skip some convolutional layers and pass directly to deeper levels, which effectively mitigates the gradient vanishing problem and ensures that the performance can be steadily improved as the depth of the network increases.

In this paper, we chose to use the ResNet18 model [32] as the core component for extracting deep local features for a detailed analysis of the intrusion detection dataset.ResNet18 belongs to the lighter models in the ResNet family, containing 18 layers of depth, which can effectively extract key features of the data while ensuring relatively low computational costs. Thanks to the design of residual blocks, the ResNet18 model passes the gradient directly through constant mapping as the network depth increases, thus avoiding the common problem of performance degradation during deep network training. This structural design of ResNet18 not only improves the model’s performance in deep learning tasks but also provides powerful support for handling complex data analysis tasks, especially intrusion detection. In the application of intrusion detection, ResNet18 provides powerful technical support. In intrusion detection applications, ResNet18 is able to dig deeper into complex patterns and local features in the dataset, which is crucial for improving the accuracy of identifying malicious behavior. By feeding the intrusion detection dataset into the ResNet18 model, the model is able to learn an abstract representation between basic network traffic features to complex attack patterns, thus improving the detection capability and accuracy of the intrusion detection system.

Network traffic datasets typically exhibit a category imbalance, i.e., the amount of data in different categories varies significantly. This leads to a tendency for neural network models during training to prioritise the identification of more numerous classes while ignoring the less numerous ones. This phenomenon only reduces the model’s ability to recognize a critical few classes, but it also affects the overall accuracy and effectiveness of the model in real-world scenarios. In view of this, this study introduces an innovative loss function specifically designed to mitigate the performance loss caused by category imbalance, thereby improving the efficiency of the model in identifying minority classes of traffic, which is particularly critical for maintaining network security.

The cross-entropy loss function is a popular method used in machine learning and deep learning for comparing the difference between the probability distribution predicted by a model and the probability distribution of the actual labels, and is often used especially when dealing with classification tasks. Its mathematical expression is used for binary classification problems. Then: (2)lossA=−1N∑i=1N[yi*log(pi)+(1−yi)*log(1−pi)]$$los{s_A} = \: - \frac{1}{N}\sum\limits_{i = 1}^N {\left[ {{y_i}*\log \left( {{p_i}} \right) + \left( {1 - {y_i}} \right)*\log \left( {1 - {p_i}} \right)} \right]}$$

where N denotes the number of samples, y_i denotes the true label category (0 or 1) of sample i, and p_i denotes the probability that the model predicts that sample i belongs to the label corresponding to γ_i. The mathematical expression for multiclassification is shown in Eq: (3)lossB=−1N∑i=1N∑c=1Cyi,c*log(pi,c)$$los{s_B}\: = \: - \frac{1}{N}\sum\limits_{i = 1}^N {\sum\limits_{c = 1}^C {{y_{i,c}}} } *\log ({p_{i,c}})$$

where if the true category of sample i is c then y_i, c = 1 otherwise y_i, c = 0, p_i,c represents the probability of predicting sample i as a c category.

KL scatter is a measure of the difference between two probability distributions P and Q. It represents the amount of loss when approximating a probability distribution P with a probability distribution Q and the mathematical expression is shown in equation (4). It is worth mentioning that the KL scatter is not symmetric, i.e., D_KL(P||Q) ≠ D_KL(Q||P). equation is: (4)DKL(P,Q)=∑c=1CP(c)*logP(c)Q(c)$${D_{KL}}(P,Q) = \sum\limits_{c = 1}^C P (c)*\log \frac{{P(c)}}{{Q(c)}}$$

Where P(c) denotes the predictive distribution and Q(c) denotes the target distribution.

This paper combines the advantages of the above loss function to construct the CEKL loss function. The basic idea is to add a penalty term after the loss. The value of the penalty term is derived from the predicted probability and the real data distribution. The CEKL loss function is shown in Eq: (5)LossCEKL=−∑c=1Cyi,c*log(pi,c)+λ*∑c=1CP(c)*logP(c)Q(c)$$Los{s_{CEKL}} = - \sum\limits_{c = 1}^C {{y_{i,c}}} *\log ({p_{i,c}}) + \lambda *\sum\limits_{c = 1}^C P (c)*\log \frac{{P(c)}}{{Q(c)}}$$

Where y_i, c = 1 otherwise y_i, c = 0 if the true category of sample i is c, p_i,c denotes the probability of predicting sample i as a c category, P(c) denotes the true data distribution while Q(c) denotes the distribution predicted by the model, and λ is a trade-off factor that balances the cross-entropy loss with the weight of the KL scatter in the total loss. By adjusting the value of λ, the model’s balance between learning the original labelling information of the data and staying close to the target distribution can be controlled.

The Softmax function [33] is an activation function that is widely used in deep learning, especially when dealing with multi-class classification problems. It is capable of converting a vector containing K real values into another vector of the same length, where the value of each element lies between 0 and 1, and the sum of these values is 1. In this way, each of the converted elements can be interpreted as a probabilistic value, reflecting the likelihood of each category. Regardless of the size of the input value, whether positive, negative, or zero, softmax maps it to a range of 0 to 1, where smaller or negative input values are converted to lower probabilities, and larger input values are converted to higher probabilities, but all probability values add up to always equal one.

Given that the softmax function outputs a probability distribution, it is sometimes called the softmax function or multicategory logistic regression. In essence, the softmax function can be viewed as an extension of logistic regression (or the Sigmoid function) for multi-category classification problems. When performing classification operations using the softmax function, it is important to ensure that the categories are mutually exclusive.

In many multilayer neural network architectures, the penultimate layer usually outputs a set of real scores that represent the scores of different categories, but it may not be convenient to use these scores directly for classification or other tasks. In this case, the softmax function is particularly important because it is able to convert these scores into a normalized probability distribution. This not only simplifies the interpretation of the output but also facilitates its use as input for other systems. Therefore, in multilayer neural networks, a softmax function is usually added to the last layer in order to convert the output of the network into a probability distribution, thus accomplishing the task of multi-class classification of the input data. To wit: (6)z=[z1,z2,...,zn]$$z = [{z_1},{z_2},...,{z_n}]$$

The softmax function is defined as follows: (7)softmax(zi)=ezi∑j=1nezj$${\text{softmax}}({z_i}) = \frac{{{e^{{z_i}}}}}{{\sum\limits_{j = 1}^n {{e^{{z_j}}}} }}$$

Where e is the base of the natural logarithm (Euler number), z_i is the i rd element of the input vector, and the denominator is the sum of the exponents of all the elements in the input vector.

The softmax function ensures that the output values are positive and normalised so that they can be interpreted as probabilities. In classification tasks, the class with the highest probability based on the softmax output is usually selected as the predictor class. In conclusion, softmax is a fundamental component in machine learning for generating probability distributions from raw scores, which is crucial for making decisions in the problem of classifying web-invasive data.

Testing network intrusion detection systems using empirical methods requires a large number of test sample instances, which together form the attack sample library. For each class of typical attacks, the sample library contains a certain number of instances, and some of these instances are used for the training of the deep learning network classification engine in order to build up a recognition model for that class of attacks inside the deep learning network Some of these instances are used for the testing of the network classification engine to check the actual working effect of network intrusion detection. The network attack sample base for this test is organised as shown in Table 6. The composition of the attack sample base is partly derived from Snort’s rule base and partly from message data captured during simulated attacks with the attack tool.

The experimental environment uses a K7 processor, 192 megabytes of RAM, and a Windows 10 operating system. 200 typical samples were collected for each type of attack behavior separately, of which 120 were used for training and 80 for testing. The proposed network intrusion detection system was used to test the intrusion of the system using malicious attack samples from the network. The network intrusion detection system should not only have the ability to identify the known attack types but also can identify the unknown attack types, and of course, it may also generate false alarms for normal behaviours. The results of the network intrusion detection system’s test results for the various metrics of the known and unknown attack types are shown in Table 7. For the known type of attack network samples, the network intrusion detection system in this paper has a high recognition rate (94.41%-97.92%), while the false alarm rate (1.02%-3.71%) and the missed alarm rate (0.01%-2.34%) are both very low. For unknown attack types, the system also has a certain recognition ability (89.34%-91.39%), which, to a certain extent, overcomes the defects of the rule-based intrusion detection system that can only detect self-knowledgeable attack behavior. It is very meaningful in the current situation where the network attack methods are being renovated very quickly. Compared to traditional intrusion detection technology, the system in the form of deep learning to achieve the memory of the attack pattern, without the need for rule base establishment, updating, matching and other management issues, thus having a high recognition efficiency, can better meet the real-time requirements of the intrusion detection system. Of course, as a preliminary implementation of the system prototype, this system still has problems such as limited detection ability, certain omissions and false alarms, which are related to the weaknesses of the convolutional neural network itself, as well as the problems of the intrusion detection system itself. However, in general, the practical effect of applying deep learning techniques to network intrusion detection systems is good, which lays a better foundation for further research on this issue.