Signal Processing and Transmission Quality Improvement Strategies in Artificial Intelligence-Assisted Software Radio Systems

The input variable’s size is determined by the number of neurons in the input layer, which can be categorized and learned using gradient descent.

The computation of the neural network update parameters can be expressed as: (1) ω=ω−ηf(ω) \[\omega =\omega -\eta f(\omega )\] where ω denotes the parameters (weights) to be updated, η denotes the learning rate, and f(ω) denotes the gradient ∂L∂ω$\frac{\partial L}{\partial \omega }$ of the loss function L(ω) about ω.

In practice, the base batch gradient descent (BGD) algorithm calculates the entire input dataset gradient, however, only one update will be performed, and the gradient of the loss function with respect to ω is calculated as follows: (2) f(ω)=1n∇ω∑i=1nL(f(x(i);ω),y(i)) \[f(\omega )=\frac{1}{n}\nabla \omega \sum\limits_{i=1}^{n}{L}(f({{x}^{(i)}};\omega ),{{y}^{(i)}})\] where L(ω) represents the loss function, n denotes the number of training samples in the dataset, and x⁽ⁱ⁾ corresponds to an objective of y⁽ⁱ⁾. Each iteration derives a derivative for the parameters in the loss function, and updates the parameters in the inverse direction according to the gradient ω.

In general, before the model is trained, the input data needs to be preprocessed in a standardized way, especially if the network contains a fully connected layer, and the input must be unified in terms of dimensional units. The main purpose of this step is to ensure that the convolutional neural network model is able to do the work that needs to be done more efficiently and accurately, otherwise it may have an impact on the performance of the entire neural network. If the dimensions of the input data are different, it can cause the convolutional neural network to converge slowly, converge less efficiently, and take too long to train. If the range of the input data is different, the data with a large relative range will occupy a larger proportion in the training process, and this situation will cause the neural network model to ignore the role of other data, and it is easy to generalize. Secondly, the value range of the input data is determined after the data needs to be pre-processed first to prevent incomplete data information.

In convolutional neural networks, the convolutional layer is an essential and important part. It is the core building block of the convolutional neural network. The convolution kernel occupies a very central position in the computation of the convolutional layer. The convolutional layer extracts the features of the input data by performing convolutional operations, and the convolutional kernel moves on the input feature surface.The more layers there are, the more complex the features can be extracted.The deeper the convolutional neural network is, the more complex the computation and the better the learning ability. However, too deep networks may be more prone to overfitting phenomena. In addition, the sliding step of the convolutional kernel is also an important parameter to consider. In general, it is important to ensure that the convolution kernel moves according to the specified sliding step just enough to convolve the entire feature surface. Otherwise, additional processing of this structure is required.

The output of each layer of the convolutional neural network is as follows: (3) Xin=AF(Uin) \[X_{i}^{n}=AF(U_{i}^{n})\] (4) Uin=∑i∈MiXin−1*kiji + bin \[U_{i}^{n}=\sum\limits_{i\in {{M}_{i}}}{X_{i}^{n-1}}*k_{ij}^{i}\;+\;b_{i}^{n}\] where AF(·) denotes the activation function used in the neural network, Xin\[X_{i}^{n}\] denotes the output feature map of the ith channel of the convolutional layer n, uin$u_{i}^{n}$ denotes the net activation of the ith channel of the convolutional layer n. uin$u_{i}^{n}$ is obtained from the output feature map Xin−1\[X_{i}^{n-1}\] of the previous layer (which may not be a convolutional layer) by convolutionally summing it with the convolutional kernel weight matrix kiji$k_{ij}^{i}$ and adding a bias bin$b_{i}^{n}$. where M_i denotes the subset of input feature maps used to compute the net activation and the “*” symbol indicates that a convolution operation was performed.

The main role of the pooling layer is to carry out secondary feature extraction without losing important information in the neural network, effectively reducing the network complexity by decreasing the data dimensions, so that it can obtain the ability to resist distortion. The output of the pooling layer still needs the activation function for rectification, and the net activation of the ind channel of the pooling layer n can be expressed as: (5) Uin=βinPF(Xin−1)+ bin \[U_{i}^{n}=\beta _{i}^{n}PF(X_{i}^{n-1})+\;b_{i}^{n}\] where βin$\beta _{i}^{n}$ represents the weight coefficient of the ird channel of the pooling layer n, bin$b_{i}^{n}$ represents the bias of the pooling layer, and PF(·) represents the pooling method used.

The fully connected layer is used to connect all the features and send the output values to the classifier. Generally, the fully connected layer is located at the end of the convolutional neural network, and can be one or more in number, and performs processes such as regression integration of features previously extracted by transforming and mapping through the convolutional and pooling layers. Similarly, in the fully connected layer, an activation function is also added, and here the ReLU function is generally used. The output Xⁿ of the fully connected layer n of a convolutional neural network can be expressed in the following two equations: (6) Xn=AF(Un) \[{{X}^{n}}=AF({{U}^{n}})\] (7) Un=ωnXn−1+bn \[{{U}^{n}}={{\omega }^{n}}{{X}^{n-1}}+{{b}^{n}}\] where the Uⁿ common to both equations represents the net activation of the fully connected layer n, which is computed by weighting ωⁿ and biasing bⁿ the output feature map X^n–1 of the previous layer in the network. ωⁿ represents the weighting coefficients of the fully connected layer n and bⁿ represents the bias of the fully connected layer n.

After processing by the fully connected layer, the input feature data is converted in dimension and the output becomes a one-dimensional vector to perform advanced decision making.

At the end of the last fully connected layer in the convolutional neural network, its output features are passed to the output layer.

Markov Decision Process (MDP) is the basis of reinforcement learning. In Markov Decision Process, the next moment state of the system is only affected by both the current moment state and the action taken at the current moment, independent of other actions or other states.

A Markov decision process can be expressed in terms of a quaternion (S,A,R,P). In the quaternion, S denotes the state space in which the intelligent body is able to switch states, A denotes the action space available for the intelligent body to make action choices, R denotes the rewards that the intelligent body is able to obtain by choosing to perform an action, and P denotes the probability of the intelligent body transferring between states. In the Markov decision process, the states and actions of the system prior to the current moment do not affect the transfer probability P and the reward R obtained by executing the corresponding action at this moment. So the transfer probability P and the reward R, can be expressed as: (8) Pss′a=P[St+1=s′|St=s,At=a] \[P_{s{{s}^{\prime }}}^{a}=P[{{S}_{t+1}}={{s}^{\prime }}|{{S}_{t}}=s,{{A}_{t}}=a]\] (9) Rsa=E[Rt+1|St=s,At=a] \[R_{s}^{a}=\text{E}[{{R}_{t+1}}|{{S}_{t}}=s,{{A}_{t}}=a]\]

The strategy of an intelligent body is generally denoted by π. The likelihood of an intelligent body performing action a in state s is represented using the conditional probability function π(a|s), which is denoted as: (10) π(a|s)=P[At=a|St=s] \[\pi \left( a\left| s \right. \right)\text{=}P[{{A}_{t}}=a\left| {{S}_{t}}=s \right.]\]

Define the cumulative reward of the intelligent body after executing strategy π at moment t until the end of the action as G_t. The cumulative reward can be expressed as: (11) Gt=∑k=0∞γkrt+k+1,γ∈(0,1) \[{{G}_{t}}=\sum\limits_{k=0}^{\infty }{{{\gamma }^{k}}}{{r}_{t+k+1}},\gamma \in (0,1)\] where γ is the discount factor, which is used to perform a weighting operation on rewards at different moments.

Since the goodness of a strategy cannot be expressed through instantaneous rewards, a form of value function that expresses the long-term impact is derived to indicate the goodness of the strategy.

Markov decision process value functions can be generally categorized into the following two types:

1) State value function V_π(s), whose expression is in the form of Bellman’s equation with: (12) Vπ(s)=E[Gt|St=s]=Eπ[ ∑k=0∞γkrt+k+1|St=s ] \[{{V}_{\pi }}(s)=\text{E}[{{G}_{t}}|{{S}_{t}}=s]={{\text{E}}_{\pi }}\left[ \sum\limits_{k=0}^{\infty }{{{\gamma }^{k}}}{{r}_{t+k+1}}|{{S}_{t}}=s \right]\] (13) Vπ(s)=E(rt+1+γVπ(St+1)|St=s) \[{{V}_{\pi }}(s)=\text{E}({{r}_{t+1}}+\gamma {{V}_{\pi }}({{S}_{t+1}})|{{S}_{t}}=s)\]

The Q-learning algorithm is a time-differentiation algorithm based on value iteration, which is widely used in non-deterministic Markov decision-making processes. This algorithm does not depend on the environment model, i.e., the learning and convergence of the optimal policy π^* can still be accomplished when the environment model is not known.

In the Q-learning algorithm, the intelligence achieves an optimal strategy by continuously increasing its value in each state, and its ultimate goal is to find a mapping from state-action pairs to Q-values [22]. The result can be represented by a matrix with N_s×N_a, N_s denoting the number of states s and N_a denoting the number of actions a that can be taken. In the Q-learning algorithm, the Bellman equation is replaced by an iterative process in which, at each time slot, the reward for action a taken in state s is calculated and the corresponding Q(s,a) is updated with the following update formula: (16) Qn+1(sn,an)←Qn(sn,an)+α(rn+γmaxaQπ(sn+1,a)−Q(sn,an)) \[{{Q}_{n+1}}({{s}_{n}},{{a}_{n}})\leftarrow {{Q}_{n}}({{s}_{n}},{{a}_{n}})+\alpha \left( {{r}_{n}}+\gamma {{\max }_{a}}{{Q}_{\pi }}({{s}_{n+1}},a)-Q({{s}_{n}},{{a}_{n}}) \right)\]

It can also be expressed as: (17) Qn+1(sn,an)←(1−α)Qn(sn,an)+α(rn+γmaxaQπ(sn+1,a)) \[{{Q}_{n+1}}({{s}_{n}},{{a}_{n}})\leftarrow (1-\alpha ){{Q}_{n}}({{s}_{n}},{{a}_{n}})+\alpha \left( {{r}_{n}}+\gamma {{\max }_{a}}{{Q}_{\pi }}({{s}_{n+1}},a) \right)\] Where, α ∈ [0,1] is the learning rate, the larger the learning rate α, the less retention of effects for previous learning. γ ∈ [0,1] is the discount factor, which is used to balance the weights of short-term and long-term gains. Q_n(s_n,a_n) denotes the value of Q at the current moment, s_n+1 denotes the next state to which the action a_n is transferred by performing the action in the s_n state. r_n denotes the reward obtained by performing the action a_n in the current state, r_n + γmax_aQ_π(s_n+1,a) denotes the estimated reward value for this Q function, and Q_n + 1(s_n,a_n) denotes the updated Q value.

The execution is repeated until the intelligence reaches the target state before stopping this training cycle.

Fading channel is a channel model that simulates wireless communication signals that experience multipath effects while propagating. In the wireless signal propagation process, the signal propagates from the transmitter to the receiver through multiple paths, which may have different propagation distances, reflections, scattering, and diffraction, resulting in a change in the received signal, i.e., multipath effect.

The channel considered in this paper is a three-path channel, and the main path is the Rice channel, which is characterized by the existence of a strong direct path in the transmission of the main path, and the rest of the paths are consistent with the Rayleigh distribution, which is the distribution of the modulus of the complex Gaussian distribution X+iY with zero-mean and independently homogeneous distributions in both the real and the imaginary parts, and the probability density function of which is: (18) fz(z)=zσ2e−z22σ2 \[{{f}_{z}}(z)=\frac{z}{{{\sigma }^{2}}}{{e}^{-\frac{{{z}^{2}}}{2{{\sigma }^{2}}}}}\] where Z is the mode of the complex Gaussian distribution X+iY and σ² is its variance. The Rice distribution represents the general case of the Rayleigh distribution, where the means of X and Y are no longer zero and are taken as m_i and m₂, respectively, and its received signal is a superposition of the complex Gaussian signal and the direct component, and the probability density function of the Rice distribution is: (19) fX(x)=xσ2I0(sxσ2)e−x2+s22σ2 \[{{f}_{X}}(x)=\frac{x}{{{\sigma }^{2}}}{{I}_{0}}\left( \frac{sx}{{{\sigma }^{2}}} \right){{e}^{-\frac{{{x}^{2}}+{{s}^{2}}}{2{{\sigma }^{2}}}}}\] where s=m12+m22$s=\sqrt{m_{1}^{2}+m_{2}^{2}}$, I₀(·) are the zeroth-order Bessel functions for the first type of correction. The Rice factor is: (20) K=s22σ2 \[K=\frac{{{s}^{2}}}{2{{\sigma }^{2}}}\]

The signal passing through the Rice channel can be represented as: (21) r(t)=h(t)os(t)+n(t) \[r(t)=h(t)os(t)+n(t)\] Where, r(t) is the received signal, h(t) is the channel coefficient, s(t) is the transmitter output signal and n(t) is the noise. The specific parameters used for subsequent simulations are shown in Table 1.

The interference to wireless communication signals during transmission can be categorized into intentional and unintentional interference. Unintentional interference mainly consists of two kinds, one is the interference caused by natural climatic phenomena in the transmission process, and the other is the interference unintentionally generated by other electronic equipment. Natural sources of electromagnetic interference mainly include lightning, solar activity, cosmic rays, and others. Interference unintentionally generated by other electronic equipment. This mainly comes from the fact that when multiple devices are operating simultaneously in the same environment, these devices may be operating in the same frequency band, resulting in interference with each other. This type of interference can usually be considered in the design phase of application scenarios, in advance of the treatment, to reduce the impact of each other.

Intentional interference mainly refers to the enemy specifically generated to disrupt the normal communication of the interference signal. Among them, deceptive interference is to simulate the real signal in the communication process of the other party, so that the other party’s receiver receives the wrong information to interfere with its normal reception. Suppression jamming is the launch of high-power jamming signals capable of covering the target signal, if you know the time the enemy sends the other side as well as the frequency band of the signal transmission, you can generate jamming signals in the time or frequency domain, the target signal suppression, the target signal submerged in the jamming, as a way of interfering with its normal reception.

The next five common signal interferences considered in this paper are described in detail.

In complex electromagnetic environments where signals are subjected to strong interference, signal classification techniques are required to extract the target signal from the mixed signals. Single-channel mixed signal classification is a key problem in the field of wireless communications. It involves recovering multiple independent signal sources from a single received signal. In this process, Bayesian models are widely used for their ability to provide a natural probabilistic framework to characterize the uncertainty and complexity of signals. Through the minimum mean square error (MSE) or maximum a posteriori probability (MAP) criterion, the a posteriori probability distribution of the unknown variables can be estimated to achieve effective classification of signals.

In this paper, through artificial intelligence techniques, the trained reinforcement learning model is used to learn the prior distribution of the signal, which is used to calculate the gradient of the MAP objective function in the classification algorithm, and the Q-learning algorithm is used to solve the objective function, estimate the target signal, and realize the signal classification.

In order to make full use of the features of the modulated signal in different transform domains to enhance the ability of the convolutional neural network model to eliminate noise, this paper proposes a noise reduction algorithm that fuses the features of the signal in the time domain and time-frequency domain, using two convolutional neural networks to extract the deeper features of the time-domain signal and time-frequency signal respectively, and achieve more effective noise reduction through feature fusion.

First, the time-domain signal to reduce noise is preprocessed. The time-domain IQ signal is transformed into the real and imaginary part of the time-domain dual-channel real signal, the time-domain one-dimensional signal is transformed into a two-dimensional time-frequency map by discrete short-time Fourier transform (DSTFT), and the complex time-frequency map is transformed into the real and imaginary part of the dual-channel real time-frequency map.

Next, the preprocessed data are fed into two different branches, the time domain encoder branch with the IQ signal as input and the time-frequency domain encoder branch with the time-frequency spectrogram obtained after the passage of the IQ signal as input.

Finally, the fused features are fed into the convolutional layers of the two branches in the time domain and time-frequency domain, respectively, to recover the signal after noise removal.

The noise suppression performance is measured using the signal-to-noise ratio gain (SNRG) and error vector magnitude (EVM), with SNRG defined as the difference in SNR before and after noise reduction, which provides a visual measure of the signal quality. Calculation formula: (26) SNRGdB=10lg(1N∑i=1N| si |21N∑i=1N| si−sin |2−1N∑i=1N| si |21N∑i=1N| si−si′ |2) \[SNR{{G}_{dB}}=10\lg \left( \frac{\frac{1}{N}\sum\limits_{i=1}^{N}{{{\left| {{s}_{i}} \right|}^{2}}}}{\frac{1}{N}\sum\limits_{i=1}^{N}{{{\left| {{s}_{i}}-s_{i}^{n} \right|}^{2}}}}-\frac{\frac{1}{N}\sum\limits_{i=1}^{N}{{{\left| {{s}_{i}} \right|}^{2}}}}{\frac{1}{N}\sum\limits_{i=1}^{N}{{{\left| {{s}_{i}}-s_{i}^{\prime } \right|}^{2}}}} \right)\] where s denotes the noise-free modulated signal, s^″ denotes the modulated signal after noise reduction, and s^′ denotes the modulated signal before noise reduction.

The error vector magnitude can be a comprehensive measure of the amplitude error and phase error of the modulating signal, defined as the ratio of the mean square deviation of the error vector signal to the mean square deviation of the ideal signal, and expressed as a percentage. The calculation method is: (27) EVM=1N∑i=1N| si−si′ |2×100% \[EVM=\sqrt{\frac{1}{N}\sum\limits_{i=1}^{N}{{{\left| {{s}_{i}}-s_{i}^{\prime } \right|}^{2}}}}\times 100%\]

In the presence of complex electromagnetic interference in software radio communication systems, estimation methods that only consider signal and noise are not directly applicable, but data-assisted artificial intelligence-based models can be applied to channel parameter estimation in interference environments.

In the environment with complex electromagnetic interference, the sequence model of the signal at the receiving end of the software radio system after sampling can be expressed as: (28) r(n)=a⋅s(n)+w(n)+J(n) \[r(n)=a\cdot s(n)+w(n)+J(n)\] where r(n) denotes the received signal, s(n) is the data signal, a is the attenuation caused by passing through the channel, w(n) is the noise, and J(n) is the interference signal. When the data signal is a known sequence of mode 1, the following equation can be obtained: (29) E(r(n)s(n))=a+(E(w(n))+E(J(n)))E(s(n)) \[E(r(n)s(n))=a+(E(w(n))+E(J(n)))E(s(n))\] (30) E(r(n))=aE(s(n))+(E(w(n))+E(J(n))) \[E(r(n))=aE(s(n))+(E(w(n))+E(J(n)))\]

The association immediately yields an estimate for attenuation a: (31) a^=E(r(n)s(n))−E(r(n))E(s(n))1−E2(s(n)) \[\hat{a}=\frac{E(r(n)s(n))-E(r(n))E(s(n))}{1-{{E}^{2}}(s(n))}\]

Once the attenuation estimate is obtained, the data signal can be stripped from the received sequence: (32) x(n)=r(n)−a^s(n) \[x(n)=r(n)-\hat{a}s(n)\] Where x(n) is the set of noise and interference signals. The sequence X(k) is obtained by applying FFT to x(n) and sorting it according to amplitude, and by dividing X(k) into m segments, each of which is l_i,i = 1,2,…,m long, the power value of each segment of the sequence can be calculated as follows: (33) Pi=∑j=0li−1| X(j) |2li,i=1,2,...,m \[{{P}_{i}}=\frac{\sum\limits_{j=0}^{{{l}_{i-1}}}{{{\left| X(j) \right|}^{2}}}}{{{l}_{i}}},i=1,2,...,m\]

Since the Gaussian white noise power bands are independent of each other and the spectrum is flat, the power of the undisturbed band is smaller than that of the disturbed band, so the smallest k segments are selected from the m segment power and averaged as the noise power, and the interference power is the x(n) power minus the noise power. The number of selected segments will have an impact on the estimation of noise power, the number of small segments will make the estimate low, the number of segments will make the estimate high, based on the typical interference bandwidth in this paper is not more than half of the total bandwidth, this paper selects m/ 2 segments as the noise power.

In summary, the overall flow of channel parameter estimation in a complex electromagnetic interference environment is shown in Figure 2.

Figure 2.

Channel estimation process under complex electromagnetic interference environment

In the experiments in this chapter, the main equipment used is the software radio system USRP RIO and the personal host PC.In this experiment, the basic frequency conversion, analog-to-digital/digital-to-analog conversion, and RF driver functional modules use the toolkit that comes with the USRP RIO platform, and the rest of the functional modules are implemented according to the design in this chapter.

The communication system for this experiment operates in the 800MHz spectrum range and divides this 800MHz spectrum into eight 100MHz wide bands. In order to better demonstrate the ability of the designed system to learn a good band selection strategy, single-tone, multi-tone, and swept interference signals are set up in the experiment, where the swept interference signal can scan the 800MHz wide spectrum within 40 seconds. The program for the software part of the experiment was designed and written using Labview2015, and the codes for the PC side and FPGA side of the user, the jammer, the information processing center, and the Q-learning cognitive engine were written respectively. The test band range used for the test system of this experiment is 2.15GHz-2.85GHz. The transmit gain and receive gain at the user side are 0dBm, and the transmit gain at the jammer side is 10dBm.

The deployment diagram of the experimental system is shown in Figure 3. Due to the limited conditions of laboratory equipment, a user (consisting of a transmitter and a receiver), a jammer, a Q-learning cognitive engine (containing convolutional neural network and reinforcement learning modules, signal classification, noise suppression and channel estimation modules, information processing nodes, and a number of perception nodes) are set up in the experiments in this chapter. During the experiment, the jammer program is adjusted so that it interferes with the user communication with a certain jamming strategy as a way to test the signal decoding, BER reduction, and anti-jamming capabilities of the software radio system.

Figure 3.

Experimental deployment of anti-jamming system based on Q learning

For the electromagnetic interference model covered in Section 2.4, the above interference signals under certain signal-to-noise ratio conditions are sequentially added to the Gaussian channel in 18.5kbps rate mode for the frequency points within the range of the base hopping frequency band to analyze the hard anti-interference capability of the test system and the performance improvement of the link after interference detection and suppression have been added. The false alarm probability of the link under several thresholds is shown in Table 2, and this paper takes the false alarm rate below 10⁻⁵ as the criterion, and chooses Th=12 as the threshold for the subsequent experimental simulation.

1)

Single-tone interference

In order to obtain the anti-interference performance of the system, one frequency point is randomly selected from 50 frequency hopping points for interference during simulation, and 1000 frames of simulation tests are performed for each signal-to-noise ratio (SNR) test point in the frequency hopping mode, and the obtained performance curves are shown in Fig. 4. It can be seen that the performance of the non-smooth waveform communication link degrades greatly when there is single-tone interference in the communication band. When the single-tone interference exceeds 20 dB, the performance degrades substantially and the BER is around 0.11. However, the BER does not deteriorate significantly again when the interference power is increased further. Because the frequency of single-tone interference is always limited, when the signal frequency hopping to the interference frequency will be completely covered and make the link performance deterioration, but when the frequency hopping to the uninterfered frequency, after filtering, will not cause too much impact on the communication link.

Figure 4.

The 18.5kbps rate mode system anti-single tone interference capability

2)

Multi-tone interference

In the multi-tone interference mode, in order to get the system anti-jamming performance, 5 frequency points are randomly selected from the 50 frequency hopping points of the communication link for interference during the simulation, and 1000 frames of simulation tests are performed for each SNR test point in the frequency hopping mode, and the performance curves obtained are shown in Fig. 5. It can be seen that when multi-tone interference exists in the communication band, the performance of the non-smooth waveform communication link degrades to a great extent. When 10 dB of multi-tone interference is applied and the SNR is -14 dB, the performance deteriorates by 1.91 dB.When the multi-tone interference exceeds 20 dB, performance deteriorates considerably, and it is almost impossible to receive and demodulate normally. Because the applied multi-tone interference itself is in the frequency hopping frequency point, when the power is low, the system can still “hard resistance”. But when the power increases to 20dB, the system is unable to receive normally.

Figure 5.

The 18.5 kbps rate mode system hard resistance to multi-tone interference capability

3)

Partial band interference

In the partial-band interference mode, in order to obtain the system anti-jamming performance, the partial-band interference of 500KHz bandwidth centered on the frequency points of 65MHz, 80MHz, and 95MHz is randomly selected from the basic frequency-hopping band of 60MHz-100MHz of the communication link during the simulation, in which the maximum allowable attenuation of the passband of the filter in the partial-band is 0.25dB, and the minimum allowable attenuation of the stopband is In the frequency hopping mode, 1000 frames are simulated for each SNR test point, and the anti-interference performance curve of the system is shown in Fig. 6. When the interference signal is within 10dB, the link can “resist” part of the band interference to a certain extent, and the performance regression is about 4.3dB. When the band interference signal is greater than 20dB, the link performance deteriorates more, and the system becomes difficult to receive and demodulate normally.

Figure 6.

The 18.5 kbps rate mode system hard resistance to Partial-band interference capability

In order to better validate the utility of the algorithm for real-time communication, we have conducted real-world communication anti-jamming decision-making experiments on the software radio platform USRP RIO.

The constellation diagrams of the software radio system in both uninterfered and interfered cases are shown in Fig. 7. Figure 7(a) shows the constellation diagram of the receiver in the case of no interference, we can see that the data points mapped in the constellation diagram are more concentrated in two frequency bands, without interference, at this time, the system’s modulation mode is BPSK modulation. Figure 7(b) is the constellation diagram of the receiver when it is interfered, at this time, the data points mapped in the constellation diagram are very decentralized, at this time, the quality of the user’s communication decreases seriously, and it starts to appear lagging, splash screen, and even communication interruption. After the user is interfered by the jammer, the transmission frequency of the system is frequency hopped from 2.2 GHz to 2.1 GHz, i.e., from one channel to another.

Figure 7.

Constellation diagram of a software radio system in undisturbed and disturbed conditions

Fig. 8 shows the BER comparison of the systems adopting different frequency hopping band selection schemes. Among them, Fig. 8(a) is the BER of the system adopting the scheme of determining the frequency hopping band selection, from which it can be seen that the system adopting this scheme can not change the transmission frequency band in time when it encounters interference. In this case, it will lead to an increase in the BER, and the average BER reaches to reduce the quality of communication. Fig. 8(b) shows the system BER graph of the frequency hopping band selection scheme based on convolutional neural network for signal classification and utilizing Q-learning algorithm. The application of reinforcement learning algorithm Q-learning can learn the complex electromagnetic environment model, and after encountering the interference, it can carry out the correct frequency hopping band selection according to the Q-table to realize the effective suppression of the interference frequency.

Figure 8.

Comparison of bit error rates of different frequency band selection schemes