Construction of personalized online educational resources based on deep learning in higher education self-study examination environment

Item Response Theory (IRT) is the classic cognitive diagnostic model constructed in the field of psychometrics. IRT is a probabilistic description of how item response outcomes to item answering are affected by the joint action of the student’s level of competence and the item characteristics in the form of a typical item characteristic curve [23].

The project contains three parameters: differentiation parameter a, which is the slope of the curve on, indicates the level of differentiation between different students on each test question. Difficulty parameter b, which is the position on the power scale of the curve, indicates the difficulty of the test questions. Parameter c, the probability of guessing the correct answer to the test question, is the asymptote.

Commonly used item characteristic function models are single-parameter model, two-parameter model and three-parameter model. The item characteristic function of each model is shown in equations (1)~(3):

Single-parameter model: (1) Pij(θj)=11+e−V(θj−bi) \[{{P}_{ij}}({{\theta }_{j}})=\frac{1}{1+{{e}^{-V({{\theta }_{j}}-{{b}_{i}})}}}\]

A two-parameter model: (2) Pij(θj)=11+e−Vai(θj−bi) \[{{P}_{ij}}({{\theta }_{j}})=\frac{1}{1+{{e}^{-V{{a}_{i}}({{\theta }_{j}}-{{b}_{i}})}}}\]

Three-parameter modeling: (3) Pij(θj)=ci+(1−ci)11+e−Vai(θj−bi) \[{{P}_{ij}}({{\theta }_{j}})={{c}_{i}}+(1-{{c}_{i}})\frac{1}{1+{{e}^{-V{{a}_{i}}({{\theta }_{j}}-{{b}_{i}})}}}\]

P_ji(θ_i) denotes the specific meaning of the probability that a student will answer test question j correctly when the response situation is θ_i, where v is the constant 1.7, e is the base of the natural logarithm which is lnN, i is the number assigned to the student, θ_i represents the student’s i proficiency value, j the numbering of the test question, and a_j,b_j,c_j denotes the discriminatory, difficulty, and the guessing parameter of the test j question, respectively.

DINA is a multidimensional discrete cognitive diagnostic model for item response functions defined as in equation (4), which represents the probability that student i will answer correctly on test question j.

η_ij denotes that student i completes the answer to test question j without taking into account the error rate and guessing rate, the number of knowledge points is K, and the calculation is expressed as equation (5).

For student i, the value of η_ni can be 1 only if he/she has mastered the relevant knowledge attributes of test j. When the value of η_i is 0, it means that student i has not mastered any of the relevant knowledge attributes of test j. In addition, equations (6) and (7) are used to express the specific meanings of error rate s_j and guessing rate g_i, respectively. s_i indicates the probability that a student has mastered all the knowledge attributes to be examined in the test question to be answered but still answers incorrectly, and g_i indicates the probability that a student has answered correctly without having fully mastered all the knowledge attributes to be examined in the test question to be answered, and there are only two parameters for each item in the DINA model, i.e., the s failure parameter and the g guessing parameter.

The association between test questions and knowledge points is portrayed in the DINA model by introducing the test question knowledge point association matrix Q, which is defined and labeled by experts for each test question.

In traditional cognitive diagnostic models, the interaction function between students and test items mainly relies on expert design, and the disadvantages of manual design are that it is more labor-intensive and the simplified function fails to reflect the intricate relationship between subjects and test items. However, the use of neural networks and cognitive diagnostic related technology combination provides a new idea, neurocognitive diagnostic model in the traditional cognitive diagnostic model based on the solution to this problem, it is directly from the data to learn students, knowledge and test scores of the interaction function, rather than manually designed, in the assessment of the state of knowledge and ability level of the student has a good performance.

The neurocognitive diagnostic model consists of three parts: the student’s mastery of the knowledge points, the factors in the test questions, and the model’s interaction function. The major difference between the neurocognitive diagnostic model and the initial cognitive diagnostic model is that the interaction function is composed of multiple layers of neural networks interlinked with each other. The inputs are the student’s mastery level on the test question, the R matrix of the student’s response record on the test question, and the relevance matrix Q of the test question’s knowledge points, and the output is the predicted probability of the student answering the test question correctly.

To ensure that each dimension corresponds to the student’s mastery level on the corresponding knowledge point at the end of model training, we make the following constraints.

The role of the attention layer is to compute the knowledge correlation weights between exercises and knowledge points. The inputs to the attention layer are the learner’s current exercise topic e_t and the set of knowledge points covered by the topic k_t. In order to map e_t to a continuous vector space, e_t is multiplied by the exercise embedding matrix A(d_k×|E|) to generate a d_k -dimensional exercise embedding vector v_t. where each d_k -dimensional column vector in the exercise embedding matrix A is an embedding representation of an exercise. The knowledge points covered by each exercise are labeled by the expert and stored in the set k_t. In this paper, we filter out irrelevant knowledge points and retain the knowledge points covered by the exercises through the knowledge point filter (K-Filter). Matrix R_t stores the embedding vectors of the knowledge points covered by the exercises, where each d_k -dimensional column vector in matrix R_t is the embedding vector of a knowledge point covered by an exercise. The inner product between the embedding vector v_t of the exercise and the embedding vector R_t(i) of the covered knowledge point is calculated, and the Softmax value of the inner product is calculated and stored in vector RS_t. Vector RS_t represents the knowledge-related weights between exercise v_t and the knowledge points covered by the exercise. See Equation (8) for details.

Softmax(zi)=ezi/∑jezi$Softmax({{z}_{i}})={{e}^{zi}}/\sum\limits_{j}{{{e}^{{{z}_{i}}}}}$ of them. The |K| -dimensional vector w_t is a vector of knowledge-related weights between the exercises and all knowledge points, and since the knowledge points covered by the exercises are filtered out by the knowledge filter, the model needs to put the knowledge-related weights of the knowledge points covered by the exercises into the corresponding positions of w_i. First, the |K| -dimensional all-zero vector w_t is initialized. First, the 5-dimensional all-zero vector 6 is initialized, i.e., w_t ←[0,⋯⋯,0]. After that, the weights of each dimension of RS_t are put into the corresponding position of w_t, i.e., w_i[k_i[i]]–RS_t[i], to obtain the knowledge-related weights of the exercises and each knowledge point.

Since the learner forgets the knowledge points that have not been reviewed in time during the learning period, and also forgets the knowledge in the interval between two learning sessions, the knowledge mastery embedding Mt.lv$M_{t.l}^{v}$ at the end of the last learning session is not the same as the knowledge mastery embedding MtFV$M_{t}^{FV}$ at the beginning of the current learning session. The forgetting layer rooted the factors affecting knowledge forgetting F_t to forget the knowledge mastery embedding matrix Mt.Iy$M_{t.I}^{y}$ at the end of the last study to get the knowledge mastery embedding matrix MtFV$M_{t}^{FV}$ at the beginning of the current study of the learner. inspired by the forgetting gate and the input gate in the LSTM, according to the factors affecting knowledge forgetting F_t to update the Mt−1v$M_{t-1}^{v}$, the first step is to erase the original information in the matrix , and then to write the information. For modeling forgetting behavior, the erasure process controls the decline of the learner’s mastery of the knowledge point, and the writing process controls the update of the learner’s mastery of the knowledge point [24].

A fully connected layer with a Sigmoid activation function is utilized to convert the factors F_t(i) affecting the learner’s forgetting level of knowledge point i into the corresponding memory erasure vector fe_t(i) for knowledge point i: (9) fet(i)=Sigmoid(FETFt(i)+bfe) \[f{{e}_{t}}(i)=\text{Sigmoid(}F{{E}^{T}}{{F}_{t}}(i)+{{b}_{fe}}\text{)}\]

where Sigmoid(z_i) = 1/(1+e^−z_i). The weight matrix FE of the fully connected layer is of shape (d_v + d_c)×d_v and the bias vector b_fe of the fully connected layer is of dimension d_v. The memory erasure vector fe_i(i) is a column vector of dimension d_v and all values in the vector are in the range (0, 1). A fully connected layer with a Tanh activation function is utilized to convert the factors F_d(i) affecting how much the learner has forgotten about a knowledge point i into a memory update vector fa_t(i) corresponding to the knowledge point i: (10) fat(i)=Tanh(FATFt(i)+bfa) \[f{{a}_{t}}(i)=\operatorname{Tanh}(F{{A}^{T}}{{F}_{t}}(i)+{{b}_{fa}})\]

where Tanh(z_i) = (e^z_j–e^−z_j)/(e^z_j+e^−z_j). The weight matrix FA of the fully connected layer is of shape (d_v+d_c)×d_v, the bias vector b_fa of the fully connected layer is of dimension d_v, and the memory update vector fa_h(i) is a column vector of dimension d_v. MtFV$M_{t}^{FV}$ is obtained by updating Mt−1v$M_{t-1}^{v}$ according to the memory erase vector and the memory update vector: (11) MtFV(i)=Mt−1V(i)(1−fat(i))(1+fat(i)) \[M_{t}^{FV}(i)=M_{t-1}^{V}(i)(1-f{{a}_{t}}(i))(1+f{{a}_{t}}(i))\]

The forgetting layer outputs the knowledge mastery embedding matrix at the beginning of this study MtFV$M_{t}^{FV}$. The forgetting layer can be used to model the learner’s forgetting process of different knowledge points separately, and to calculate the degree of forgetting of different knowledge points according to the difference in the learner’s learning history of different knowledge points.

After a learner takes a course, the online education system prepares questions for the learner to test the learner’s learning. The learning layer tracks the changes in knowledge acquisition during the learning process based on the learner’s answer results, and models the learning behavior by updating the knowledge acquisition embedding matrix MtFV$M_{t}^{FV}$ at the beginning of the learning process to the knowledge acquisition embedding matrix Mtv$M_{t}^{v}$ at the end of the learning process. Tuple (e_t,r_t) represents the learner’s answer result at time t. In order to map tuple (e_t,r_t) to a continuous vector space, tuple (e_t,r_t) is multiplied with the answer result embedding matrix B of size d_v*2|E| to obtain a d_v -dimensional answer result embedding vector s_t. The learning layer takes the question-answering result embedding vector s_t and the knowledge-related weight vector w_t corresponding to the exercise questions as inputs, and uses the learner’s practice results in the online education system as the indirect feedback of the learner’s learning effect, updating the learner’s knowledge mastery state in the process of learning and modeling the learning behavior through the LSTM network: (12) Mtν(i)=LSTM(st,wt(i)MtFV(i);θ) \[M_{t}^{\nu }(i)=\text{LSTM(}{{s}_{t}},{{w}_{t}}(i)M_{t}^{FV}(i);\theta \text{)}\]

As can be seen from Eq. (12), the learning layer only updates the learner’s knowledge mastery embedded in the knowledge points covered by the exercises, and the learning layer does not process the knowledge points that have nothing to do with the exercises.

The purpose of the prediction layer is to predict the learner’s performance on the next candidate exercise e_t+1. Since the learner’s forgetting behavior during the two-answer interval affects his/her knowledge mastery state, the prediction layer predicts the probability of the learner answering Exercise e_t+1 correctly based on the knowledge mastery embedding matrix Mt+1FV$M_{t+1}^{FV}$ at the start of the answer at the current moment. By weighting and summing the knowledge-related weights w_t+1 and the learner’s knowledge mastery embedding Mi+1FV$M_{i+1}^{FV}$ at the start of the current moment, the resulting vector d_n+1 is the weighted mastery embedding vector of the knowledge points covered by the exercise.

The success of the learner in answering an exercise is not only related to the learner’s comprehensive mastery of the knowledge points covered by the exercise, but also related to the exercise itself, so the combined vector [d_t+1,v_t+1] obtained by connecting vectors d_t+1 and v_t+1 is input to a fully connected layer using the Tanh activation function, and the output vector h_t+1 contains not only the learner’s comprehensive mastery of the knowledge points covered by the exercise, but also the characteristics of the exercise itself. Matrix W_I and vector b_I represent the weights and biases of the fully connected layer, respectively.

Finally, vector h_t+1 is input into a fully connected layer with a Sigmoid activation function to obtain p_t+1 which represents the probability of the learner answering the exercise correctly. Where matrix W₂ and vector b₂ represent the weights and biases of the fully connected layer respectively.

The knowledge level output layer takes the knowledge mastery embedding matrix Mtv$M_{t}^{v}$ at the end of the learner’s study as input, and outputs a |K| -dimensional vector value representing the learner’s knowledge mastery level at the end of this study, with each dimension of the vector between (0, 1), indicating the learner’s mastery level of the knowledge point.

In the prediction layer, at each time node t, the learner’s performance on a particular exercise e_i is predicted based on two inputs: the learner’s comprehensive mastery embedding vector d_t for the knowledge points covered by that exercise and the exercise embedding vector v_t. Thus, if one just wants to estimate the learner’s mastery of the i th knowledge point without any inputs from the exercise, one can omit the exercise embedding v while letting the learner’s knowledge mastery embedding i column Miv(i)$M_{i}^{v}(i)$ of matrix Miv$M_{i}^{v}$ as the input to the equation. After the output matrix Miv$M_{i}^{v}$ of the learning layer, in order to estimate the mastery level for the i th knowledge point, the weight vector β_i = (0,⋯,1,⋯,0) is constructed in which the value of the i th dimension is equal to 1, and the knowledge mastery embedding vector Miv(i)$M_{i}^{v}(i)$ for the i th knowledge point is extracted, after which the knowledge mastery level is estimated using equations (17) and (18): (16) Mtν(i)=βiMtν \[M_{t}^{\nu }(i)={{\beta }_{i}}M_{t}^{\nu }\] (17) yt(i)=Tanh(WIT[MtV(i),0]+bI) \[{{y}_{t}}(i)=\operatorname{Tanh}(W_{I}^{T}[M_{t}^{V}(i),0]+{{b}_{I}})\] (18) valuet(i)=Sigmoid(W2Tyt(i)+b2) \[valu{{e}_{t}}(i)=Sigmoid(W_{2}^{T}{{y}_{t}}(i)+{{b}_{2}})\]

Vector θ = (0,0,⋯,0) has the same dimension as the exercise embedding v_t and is used to complement the vector dimension. Parameter W_l,W₂,b_I,b₂ is exactly the same as in Eq. (14) and Eq. (15). The mastery level of each knowledge in the knowledge space is calculated sequentially to get the learner knowledge mastery vector value_t.

The parameters that the model needs to be trained are mainly the exercise embedding matrix A, the answer result embedding matrix B, the neural network weights and biases, and the knowledge matrix M^K. In this paper, we optimize each parameter by minimizing the cross-entropy loss function between the model’s prediction of the learner’s answer result p_t and the learner’s true result of answering the question r_t. The loss function is shown in Equation (19). In this paper, the Adam method is used to optimize the parameters.

In the problem of test question recommendation in education, assume that there are N student SS = {s₁,…,s_n,…,s_N} and M test questions ExS = {ex₁,…,ex_i,…,ex_M}, and the online learning interaction behavior of the student can be defined as the sequence: R = {(e₁,r₁),…,(e_t,r_t),…,(e_T,r_T)}, where e_t ∈ E_xS denotes the questions that the student s_n has done at the moment of t, and r_t denotes the corresponding scoring performance. The time state based on the time after the t moment is represented by ks_T, β_n,i denotes the difficulty for the student s_n test question ex_i, and α_n,i denotes the recommendation of test question ex_i to the student s_n.

Extensive problem training is an important part of the teaching task. The purpose of doing problems is to master the knowledge rather than doing the problems themselves, and students digest and absorb the newly learned knowledge and consolidate the knowledge and methods they have already learned through doing problems and tests. However, there are individual differences in the degree of students’knowledge mastery, and there are a large number of test materials, such as different levels of difficulty, so how to get the most suitable test questions for their own knowledge status from the huge amount of test materials, and personalized test recommendations for each student is a difficult problem.

In recent years, researchers have made many attempts and studies on test question recommendations. Collaborative filtering-based recommendation algorithm is one of the most representative recommendation methods, which recommends test questions by considering the similarity between students or exercises. However, the collaborative filtering method adopts a static view in the recommendation process, so when the recommendation environment is placed in the field of education, it ignores the dynamic characteristics of the personalized recommendation scenario, and fails to take into full consideration of the knowledge points corresponding to the test questions and the dynamically changing personalized ability differences of the students, which leads to a lack of reasonable interpretability in the recommendation results, and the students are unable to know the reasons why the test questions are recommended, and they are also unable to know their own Students cannot know the reason why the test questions are recommended, nor can they know exactly where their weak areas are, so the recommendation results may not necessarily be beneficial to students’ ability improvement. In addition, some researchers recommend test questions to students by evaluating the consistency of the content of the test questions, and some scholars recommend test questions based on students’ cognitive diagnostic ability, but these methods do not directly measure the state of students’ knowledge, but rather quantify the students’ ability as knowledge points, and the students’ cognitive status is invisible to a certain extent, and there is a certain degree of invisibility, which will lead to a relatively poor recommendation result.

Unlike traditional positive feedback recommendation algorithms based on user and item similarity, the problem of personalized test recommendation for students in an educational setting is more specific: the ultimate goal of test recommendation is not to recommend the most interesting test questions for students, but rather to consolidate their foundations and to improve their competence in the educational domain. Similarly, test recommendation is not simply a matter of recommending the hardest or easiest questions to students. The test questions that are the most challenging may be beyond the student’s knowledge base abilities, while the easiest test questions may already contain knowledge points that the student is already proficient in. In this situation described above, students face the dilemma of not being able to practice effectively with these test questions and therefore not being able to significantly improve their knowledge points. The personalized test question recommendation method based on the GFLDKT model is shown in Fig. 1, and in this paper, based on the students’ individualized ability differences, combined with the mastery of specific knowledge points, a reasonable test question difficulty is set to provide students with personalized test question recommendations [25].

Figure 1.

Personalized test recommendation method based on GFLDKT model

Specifically, the personalized test question recommendation method in this paper is able to obtain explicit student knowledge point mastery and take the probability of a student s_n correctly answering a test question ex_i containing a certain knowledge point kc_j as the student’s test question difficulty β_nji, and finally, we recommend test questions of appropriate difficulty according to the learning needs of different students. Specifically, the students’ test question difficulty β_nji is defined as follows: (20) βnji=1−r^T+1 \[{{\beta }_{nji}}=1-{{\hat{r}}_{T+1}}\]

Test question difficulty is an important component of test question recommendations. Whether a student’s focus is to fill in knowledge gaps by repeating knowledge with a low level of mastery, or to consolidate and improve knowledge with a high level of mastery in order to obtain higher grades, students with different cognitive abilities have different needs for test question difficulty, and test question difficulty boundaries for that knowledge point are set ζ_j1,ζ_j2(ζ_j1 < ζ_j2) in order to better assess the mastery of students. At the same time, according to the students’ answer prediction r^T+1${{\hat{r}}_{T+1}}$, you can create a personalized student test question recommendation ensemble ExS_rec to provide students with targeted practice.

On the other hand, with the help of massive interactive data of students’ test questions, according to the probability of students’ correctly answering the test questions, the general difficulty of the test questions is obtained, and the corresponding test question recommended mass ensemble ExO_rec is established, in order to solve the problem that some learners use the online education platform for the first time, which leads to the lack of analyzing data to get the accurate mastery of the knowledge points (learners will be mapped out before they use the platform for the first time, and the data of the mapping test is utilized to The data from the test will be used to initially obtain the students’ knowledge point mastery status ks₁).

The higher the average correct rate of a test question, the lower the difficulty of the question, so we counted the total number of times students answered the question correctly n₀₀ and the total number of times they answered the question incorrectly n₀₁ from the data set, and used the correct rate of the question to calculate the difficulty of the question ϕ_i: (22) ϕi=1−n00n02 \[{{\phi }_{i}}=1-\frac{{{n}_{00}}}{{{n}_{02}}}\] (23) ExOrecn={exi|exi∈ExOrec,ϕi∈[ζ1,ζ2]} \[Ex{{O}_{re{{c}_{n}}}}=\{e{{x}_{i}}|e{{x}_{i}}\in Ex{{O}_{rec}}\;,\;{{\phi }_{i}}\in [{{\zeta }_{1}},{{\zeta }_{2}}]\}\]

In the process of test question recommendation, the Softmax function is first used to normalize the test question representations, and then the difficulty of the test questions is used as a weighting coefficient to calculate the real response of students with different knowledge base abilities in completing the test questions with different difficulty coefficients: (24) MEi=softmax(Ei)=exp(Ei)∑j=1K(exp(Ei)) \[M{{E}_{i}}=softmax({{E}_{i}})=\frac{exp({{E}_{i}})}{\sum\limits_{j=1}^{K}{(exp({{E}_{i}}))}}\] (25) Dj=kstj−MEij×βni \[{{D}_{j}}=k{{s}_{tj}}-M{{E}_{ij}}\times {{\beta }_{ni}}\]

Finally, a threshold value of θ is set to determine whether the final test recommendation is reasonable or not, if 0 < D_j ≤ θ, it indicates that the recommendation is effective; if D_j < 0 or D_j > θ, it indicates that the recommendation is unreasonable, and the recommended test questions fail to help students consolidate their knowledge base and improve their performance.

The physical layer completes the access integration, extraction, and cleaning of data resources from each examination base, and establishes a cross-platform fusion and shared resource cloud. This layer mainly solves the association of resources across media and languages and the problem of knowledge implicitness through multi-source heterogeneous fusion management middleware; solves the problem of ubiquitous distribution of data resource nodes; solves the problem of multimodal and dynamic change of data; completes the efficient processing and semantic understanding of heterogeneous big data; hides the heterogeneous nature of the network components, realizes the transparency of the servers, the transparency of the network and the transparency of the language, and achieves the The precise and efficient integration of cross-campus resource libraries is realized.

The data layer is the core layer of the platform architecture, which manages the shared resource cloud and mainly contains three aspects:

(1) Resource management: Audit the resources that join the resource cloud, allocate, access, apply, billing management and so on according to the relevant mechanism.

The application layer mainly builds corresponding services on the basis of the data layer, which contains basic services such as basic resource base, recommendation push service, visualization report, etc. Each institution can build a personalized resource base on this basis according to its own situation, and improve and perfect the resource base of our university.In addition, it establishes application interfaces and stipulates the use agreement, access mode, and charging mode of the application to provide services to the upper layer.

The function of the service layer is mainly to complete personalized service customization by each institution according to its own needs.Through the application layer interface, it selects applications and related protocols, service modes, and other system strategies from the application library, and customizes the resource management system suitable for the university.

In order to evaluate the effectiveness of the method proposed in this paper, experiments are conducted on real data collected from 3 online learning platforms, the datasets are described as follows:

ASSIST2009: This dataset is provided by the ASSISTments online teaching platform and has been used in several papers for the evaluation of knowledge tracking models. After removing duplicate records, it contains 325,673 interaction records from 4151 students on 110 knowledge points.

ASSIST2015: this dataset collected 708,631 records from 19,917 students across 100 knowledge points. Although this dataset has more records than ASSIST2009, the average number of student interaction records is less due to the larger number of students.

ASSISTchall: This dataset collected 942,816 interaction records from 686 students across 102 knowledge points. This dataset has been used in the 2017 ASSISTments Data Mining Competition and has a higher average number of records per student.

In order to verify the validity of the model in this paper, the following seven typical benchmark models are selected to compare the experimental results:

BKT: The learning process is modeled as a Hidden Markov Process, with knowledge mastery as a binary variable of mastered and un-mastered, and learning is modeled as a discrete transition from un-mastered to mastered state of a knowledge point;

DKT: a pioneering work that introduces deep learning to knowledge tracking, using recurrent neural networks to model the student’s knowledge state at each input moment;

DKVMN: modeling students’ mastery of each knowledge point using a storage matrix, which solves the problem of DKT’s inadequacy in dealing with long sequence dependencies;

GIKT: uses Graph Convolutional Networks (GCN) to model problem-knowledge point correlations, GIKT summarizes students’ mastery of knowledge points as the interaction between students’ current state, students’ history of related exercises, the target problem, and related knowledge points;

EERNN: considers information about the semantics of the exercises and uses the semantic representations of the test questions as input to the model, taking into account the bias in prediction due to the large differences brought about by the semantics of two exercises possessing the same knowledge points;

LPKT: gives equal attention to the results of incorrect and correct answers, arguing that students will learn from them even if they answer incorrectly;

ATKT: enhances the generalization of knowledge tracking using adversarial training, proposing an aggregation module for the hidden states of knowledge while emphasizing the importance of the current knowledge state.

Referring to most of the research works on knowledge tracking models, this experiment adopts accuracy (ACC) and area under the ROC curve (AUC) as the evaluation metrics from the classification point of view, where AUC evaluates the probability that a predicted positive example is ranked in front of a negative example.

The experiment divides 80% of the data in each dataset into training and validation sets, and 20% into testing sets. The experiments are performed in stochastic gradient descent using a learning rate r ∈ {0.0001,0.001}, an embedding dimension k ∈ {100, 150, 200}, a training batch size B ∈ {100, 150, 200, 300}, a number of hidden layers h ∈ {32, 64, 128}, and the number of neurons in the hidden layer of the LSTM model is set to 200.The results of the experiments are taken on each dataset average of 10 calculations. This experiment is run on a 64-bit, 3.6GHz GPU (RTX 3060 12G) computer. Figure 3 shows the loss curve of the model trained in this paper.The model basically reaches convergence after 46 rounds of iterations, and the RMSE is stable around 0.88.

Figure 3.

Model training loss curve

The experimental results are shown in Table 1, where the black bolding is the optimal effect obtained by this dataset in the corresponding index. The experimental results show that the model proposed in this paper achieves better results on all three datasets, with ACC values of 0.823, 0.824, and 0.818, and AUC values of 0.827, 0.816, and 0.821, respectively.The two metrics are the highest among all the models, and the best results are achieved.

Although our model obtains better performance, whether the model is interpretable or not is also an important evaluation aspect of knowledge tracking, in order to justify the model tracking students’ knowledge status, we visually tracked the mastery status of five students in the same knowledge point, and the results are shown in Fig. 4. The darker colors indicate that the corresponding knowledge point has a better mastery status.It can be seen that the pace of mastery of this knowledge among different students is basically the same, which is in line with teaching practice and the law of education.The mastery status of the five students after the 30th time unit has reached more than 0.74, and the mastery status of the corresponding knowledge point has improved.Therefore, the visualization results demonstrate the rationality of the knowledge tracking process.

Figure 4.

Information status changes visualization

The model in this paper is able to capture the correlation between test questions by calculating the distance of the exercises embedded in the space, which is reflected in the knowledge and difficulty involved. The 200 exercises from the ASSIST2009 dataset were analyzed by clustering them in the knowledge space, and the results are shown in Figure 5. It can be seen that these exercises are categorized into 12 categories, and the test questions in each category may have the same knowledge points, and the closer the distance between the points represents the closer their difficulty level, and these automatically learned results can be used as a data supplement in the education domain.

Figure 5.

Information status changes visualization

In addition, this paper also analyzes the difficulty of the exercises in the three datasets. The results are shown in Fig. 6, which shows that the difficulty coefficients b of most of the exercises are distributed between 0.4 and 0.8, and the exercises of ASSISTchall are more evenly distributed in each difficulty interval relative to the other two datasets.

Figure 6.

Question analysis