A Study on Teaching Quality Improvement of Microcontroller Principles and Applications Course Based on Convolutional Neural Networks

Figure 1.

VGG16 network structure

Where m_i denotes the cumulative momentum term at moment i, w_i denotes the model parameters at moment t, η_t denotes the learning rate at moment t, and η_iΔJ (w_i, x_i, y_i) denotes the amount of updates in iteration 1. If the gradient at the current moment converges with the historical gradient direction, this trend will be strengthened at the current moment, otherwise the gradient direction at the current moment is weakened. Instead of doing the gradient update at w_i, the update is done at w_i – αm_i−1 one momentum unit ahead, resulting in the Newton’s accelerated gradient algorithm, which takes the form shown in (11): 11{ mt=αmt−1+ηtΔJ(wt−αmt−1)wt−1=wt−mt

For the classification problem of face expression recognition to be solved in this paper, the SGD optimization algorithm with Nesterov Momentum is selected as the optimizer of the model, which can ensure that the model training has a high speed and avoid falling into local minima.

In this paper, we use a channel-level pruning model compression method and a binarization-based model quantization method to achieve lightweight improvements in the VGG16 model.

Model compression is a common method for improving the efficiency of network models, specifically by compressing trained network models. On the one hand, it can be started from the perspective of compressing the model weights, and on the other hand, it can be started from compressing the network architecture. Therefore, the network can be trained and stored fewer network parameters, which solves the problem of model storage and prediction speed.

Deep learning network models have a large number of redundant parameters from the convolutional layer to the fully connected layer, and a large number of neurons with activation values converging to 0. The removal of these neurons can result in the same model representation, which is known as over-parameterization. The corresponding technique is known as model pruning. Dropout randomly sets the outputs of some neurons Q to zero, which is neuron pruning, while DmpConnect randomly sets the connections between some neurons to zero, making the weight connection matrix sparse, which is weight connection pruning.

Quantization is another common method of model compression. Quantization is the approximation of weights or activation values expressed in high bit-width (32float) with lower bit-width (int8), which is reflected in the numerical value of discretization of continuous weight values or activation values. By quantizing, it is beneficial to speed up the operation. For example, when transforming a 32-float into an int8 representation, fixed-point arithmetic is faster than floating-point arithmetic, while reducing data storage space by about 1 / 4.

The concept of attentional mechanism comes from the study of human vision, and in cognitive science, people tend to notice only a part of the information and selectively ignore the information they don’t pay attention to, a phenomenon known as attentional mechanism [29]. Usually, all parts of the human retina have different abilities to process information, which is also known as visual sensitivity. To maximize the limited visual information processing resources, it is necessary to select a specific area in the visual field and concentrate on it. For example, when a person is reading, they will pay attention to and process a small portion of the words to be read. In summary, the attention mechanism consists of two parts: one is to identify the region to focus on, and the other is to allocate the limited information processing resources to the focus region.Attention algorithm is to utilize the principle of attention resource allocation in the human brain, and utilize the probabilistic weighting method to probabilistically weight the different input features, so as to better identify the features that are relevant to the task. The basic structure of the Attention model is shown in Fig. 2.

Figure 2.

Basic structure of attention mechanism

The role of dual stream convolutional neural networks in video classification and behavioral recognition is becoming more significant [30]. In recent years, scientific researchers have conducted many experiments on dual-stream networks to improve their recognition effect and enhance their performance. In the process of behavior recognition, spatial network and temporal network input image data is different, the network structure is not exactly the same, to find the difference between the features extracted by the two networks at any time, to remove the redundancy between the features, and to synthesize the effective information, which is a complex and laborious work. With the application of the attention mechanism in other fields, researchers have noticed the powerful function of the attention mechanism, and gradually combine the attention mechanism with dual-stream convolutional networks, with the purpose of improving the recognition efficiency and accuracy of dual-stream networks.

In this paper, the attention mechanism module is added to the dual-stream convolutional neural network to form a dual-stream convolutional neural network based on the attention mechanism, which is structured as shown in Fig. 3, where the video data of students’ behaviors are collected and processed in frames, and the input information of the spatial network is screened out as a single RGB frame image, and the remaining frames are selected to be processed into a continuous 10 frames to be inputted into the temporal network as a stacked optical flow map, and the dimensions of the input images are is uniformly 224 × 224, and the main network is composed of a pre-trained VGG16 network as well as an attention module. Except for the attention module, the dual-stream convolutional neural network based on the attention mechanism is exactly the same as the traditional dual-stream network, and the next section mainly explains the combination of the attention mechanism and the dual-stream network.

Figure 3.

The double flow convolution neural network with the attention mechanism

In view of the lack of connection between the two networks and the lack of filtering of key regions in the traditional dual-stream network, this paper proposes to add an attention mechanism module to the network, embed the attention module between the convolutional layers, and use the attention weights to screen the key regions layer by layer, as shown in Figure 3, the attention module is composed of 4 Attention matrices and 3 pooling layers, and each convolutional layer of the spatial network and the time network is connected by an Attention matrix. A pooling layer is arranged after each Attention matrix, which is used for pooling operations and reduces the number of parameters. In the implementation process, the first step is to calculate the weight matrix of Attention1, the remaining three Attention matrices can be obtained after three pooling in the network, the two images are input to the spatial network and the temporal network after the output of the feature maps, the size of the image input to the network is 224 × 224 × 3, the output of the feature maps of the first convolutional layer in the two networks are cascaded in channels, cascaded The size of the feature map after cascading is 112 × 112 × 128, where the size of each feature map is 112 × 112,128 is the number of channels, i.e., the number of feature maps. Then, the cascaded feature maps are used to obtain the weights of attention with a complete connection hierarchy, which is calculated using the following formula: 12GIJ=tanh(w1tHi,j+B1) 13GIJ=egi,j∑i=1112∑j=1112egi,j

Where, w₁ is the weight, b₁ is the bias, H_ij ∈ R^l*d is the depth eigenvector at position (i, j)∈(1–112) on the feature map, g_ij is the un-normalized attention weight and G_ij is the normalized attention weight matrix. By passing the obtained attention weights g_ij through the Softmax function, the 112 × 112 normalized attention weight matrix G_ij, Attentionl, is obtained.

After calculating the first attention weight matrix Attention1, Attention1 is pairwise dot-multiplied with each feature map output from the second convolutional layer of the spatial and temporal networks, respectively, so as to obtain the feature map with attention weights, which is used as the input of the third convolutional layer group. At the same time, pooling operation is performed on the Attention1 weight matrix, and after pooling, an Attention2 matrix of size 56 × 56 is obtained, and the Attention2 matrix is multiplied by each feature map output from the third convolutional layer of the spatial and temporal networks, respectively, to obtain the feature map with the attention weight, which is the input of the fourth convolutional layer group. Attention2 is pooled to obtain an attention weight matrix Attention3 of size 28 × 28. Attention weight matrix Attention3 is multiplied by each feature map output from the fourth convolutional layer of the spatial and temporal networks respectively to obtain a feature map with attention weights as the input of the fifth convolutional layer. Attention3 continues the pooling operation to obtain an attention weight matrix Attention4 of size 14 × 14. The attention weight matrix Attention4 is pairwise dot-multiplied with each feature map output from the fifth convolutional layer of the spatial and temporal networks, respectively, to obtain the final feature map with attention weights. Finally, the output feature maps of the fifth convolutional layer group obtained from the temporal and spatial streams after the above operations are cascaded and fed into the fully connected layer for action-behavior recognition.

On this basis, by adding four attention mechanism modules, the method can not only selectively learn according to the importance of the information, eliminate redundant and interfering actions, and enhance the weight of the target, but also make full use of the correlation between temporal and spatial networks to obtain more accurate and diversified behavioral characteristics, thus improving the ability to identify students’ behaviors.

The pre-test results of the experimental (EC) and control (CC) groups’ performance in Principles and Applications of Microcontroller are shown in Table 4 and the post-test test results are shown in Table 5.

As can be seen from the pre-test and post-test comparison results in Table 4 and Table 5, the score difference between the two groups in the five dimensions of Microcontroller Principles and Applications before the experiment is 0.18, 0.31, 1.09, 0.04, 0.45, respectively, which are not significantly different, and the p-value is greater than 0.5, and the two groups have similar scores. After the teaching experiment, the P-value for all five dimensions was less than 0.5, and the achievements of the two groups created a significant gap. The experimental group outperformed the control group by 4.49, 4.37, 3.87, 5.31, and 6.24 in specification implementation, system design, system simulation, hardware extension, and experimental analysis, respectively.

The beginning and end of semester grades in Principles and Applications of Microcontrollers for the experimental and control groups, respectively, should be examined to examine the changes in the two groups before and after the experiment. The results of the pre- and post-test comparisons of the experimental group’s “Microcontroller Principles and Applications” grades are shown in Table 6, and the results of the control group’s pre- and post-test comparisons are shown in Table 7.

From the data in Tables 6 and 7, it can be seen that the experimental group’s performance in Principles and Applications of Microcontrollers after the experiment improved considerably compared to the preexperiment, with an increase of 5.38, 4.12, 4.98, 5.39, and 5.79 in five dimensions, and p-values of the dimensions in the pre- and post-tests were less than 0.05. On the contrary, the performance of the control group increased by 1.07, 0.06, 0.02, 0.12, and 0.10 on the five dimensions after the experiment, with p-values greater than 0.05, and the change in the performance of the control group was negligible, with the level of performance before and after the experiment basically at the same level.