Curriculum Content Construction and Updating for Intelligent Educational Systems Using Knowledge Graphs

Knowledge graph is a structured knowledge representation that presents information about relationships and attributes between entities in the form of a graph. Its goal is to build a complete and accurate knowledge base so that machines can understand and reason about this knowledge to better provide intelligent services to humans. Knowledge graph data is derived from a variety of structured, semi-structured and unstructured data in the Internet. These data come from a variety of information sources, and by processing and extracting these data, entities, relationships and attributes can be extracted and organized into an organic graph structure [27].

The construction methods of knowledge graph can be mainly categorized into top-down and bottom-up.

Top-down construction method: this method usually starts with constructing an ontology model, which defines the basic elements such as entities, relations and attributes of the target domain, and specifies their semantics and constraints. This is the conceptual layer for constructing a knowledge graph, also known as the ontology layer. Then, based on this ontology model, relevant entity data are extracted and organized to form the entity layer of the knowledge graph.

Bottom-up construction approach: this approach, on the other hand, collects a large amount of raw data first, and then discovers the entities, attributes and relationships in the data through data mining, information extraction and other techniques to construct the entity layer of the knowledge graph. Subsequently, the characteristics of the entity layer are analyzed and summarized, and the conceptual layer of the knowledge graph is gradually formed.

The construction of knowledge graph is based on the ontology rules defined by domain experts to extract and fuse the correct triples that match the facts from a large amount of heterogeneous data. In order to realize the accurate extraction and fast aggregation of a large amount of knowledge, the whole construction process needs to use techniques such as ontology construction, entity naming, relationship extraction, knowledge storage and so on.

The data layer originates from major platforms in the Internet, and it needs to process the raw data for cleaning and alignment. Through the method of entity recognition, meaningful entity information is identified in a large amount of heterogeneous data. In the algorithm layer, the encapsulated algorithms are invoked according to the needs of the task layer. The data are chained and finally get the standard compliant ternary format and imported into the graph database to complete the storage of knowledge. In the final task layer, this stored knowledge will be utilized for various applications such as smart quizzes. This link involves how to utilize the stored knowledge and how to answer students’ questions effectively through the smart quiz module.

Before named entity recognition, data preprocessing is needed. Upon receiving a question from a student, the question can first be segmented using jieba, a third-party library for python. jieba is a popular Chinese lexical library, which is capable of realizing the functions of word segmentation, lexical annotation, and keyword extraction of Chinese text. The result of the lexical processing is a series of words or phrases, which can describe different aspects of the problem.

In order to reduce the labor cost, here in this paper, a deep learning based approach is used and the BI-LSTM-CRF model is chosen for named entity recognition.

BI-LSTM-CRF is a deep learning structure that consists of two parts: a bidirectional long and short-term memory network (BI-LSTM) and a conditional random field (CRF), which can be used for named entity recognition. Compared with the unidirectional LSTM network, BI-LSTM can solve the problem that LSTM can only encode forward utterance information to some extent.

LSTM is a special type of RNN that learns long-term dependent information. LSTM adds cell states to the RNN C_t and designs a “gate” structure for the ability to add or remove information to cell states. A standard LSTM consists of a forget gate, an input gate, and an output gate [29]. 1)

Key Strategies for Learning Quality Experience Driven

A forget gate can selectively forget information in a cell state, which reads the hidden state h_t−1 of the previous moment and the input x_t of the current moment, outputting a value between 0 and 1 to each number in cell state C_t−1. When the output result is “1”, it means “completely retain”, and when the output result is “0”, it means “completely discard”. The output is expressed as f_t, see equation (1): 1ft=σ(Wf⋅[ht−1,xt]+bf)$${f_t} = \sigma \left( {{W_f} \cdot \left[ {{h_{t - 1}},{x_t}} \right] + {b_f}} \right)$$

New information can be selectively recorded into the cell state by the input gate, which reads the hidden state h_t−1 of the previous moment and the input x_t of the current moment, and processes them into the information brought by the new input C˜t$${\widetilde C_t}$$ and the information retained by the new information i_t, see Equation (2) and Equation (3). The updated cell state C_t can be obtained from the results of the forgetting gate and the input gate, see Equation (4): 2C˜t=tanh(WC⋅[ht−1,xt]+bC)$${\tilde C_t} = \tanh \left( {{W_C} \cdot \left[ {{h_{t - 1}},{x_t}} \right] + {b_C}} \right)$$ 3it=σ(Wi⋅[ht−1,xt]+bi)$${i_t} = \sigma \left( {{W_i} \cdot \left[ {{h_{t - 1}},{x_t}} \right] + {b_i}} \right)$$ 4Ct=ft⋅Ct−1+it*C˜t$${C_t} = {f_t} \cdot {C_{t - 1}} + {i_t}*{\tilde C_t}$$

The output gate represents the output of the LSTM, which reads the hidden state h_t−1 of the previous moment and the input x_t of the current moment to compute the result o_t of the output gate, see Equation (5). In addition, the hidden state at this moment can be calculated by combining the updated cell state C_t, see Equation (6): 5ot=σ(Wo⋅[ht−1,xt]+bo)$${o_t} = \sigma \left( {{W_o} \cdot \left[ {{h_{t - 1}},{x_t}} \right] + {b_o}} \right)$$ 6ht=ot⋅tanh(Ct)$${h_t} = {o_t} \cdot \tanh \left( {{C_t}} \right)$$

BI-LSTM combines forward LSTM and backward LSTM to obtain two different kinds of implicit layer information, sequential and reverse order, of the input sequence, which enables better modeling of sequence information and learning of contextual semantic information.

In the named entity recognition task, BI-LSTM can effectively encode the information in the sentence, better recognize bidirectional semantic dependencies, and improve the accuracy of entity recognition. 2)

Learning Quality Experience-Driven Critical Strategies

The CRF model is a discriminative model for modeling conditional distributions. It takes into account the influence of neighboring contextual information or states on the prediction, and is more suitable for named entity recognition. The output of BI-LSTM for each word is a labeling score, and CRF can add some constraints to the final constrained labels to ensure the validity of the predicted labels [30]. The loss function of CRF has two types of scores, the emission score and the transfer score: (1)

The emission score can be obtained from the labeling scores output from BI-LSTM.

The goal of the model is to maximize the probability of the true path, the representation of which is shown in Equation (7). Where P_RealPath denotes the score of the true path and P₁ + P₂ + … + P_N denotes the score of all possible paths: 7prob=PRealPathP1+P2+⋯+PN$$prob = \frac{{{P_{\operatorname{Re} alPath}}}}{{{P_1} + {P_2} + \cdots + {P_N}}}$$

For ease of calculation, logarithmic and negative numbers are taken from Eq. (7) as the loss function, see Eq. (8): 8LossFunction=log(P1+P2+⋯+PN)−logPRealParh$$LossFunction = \log \left( {{P_1} + {P_2} + \cdots + {P_N}} \right) - \log {P_{\operatorname{Re} alParh}}$$

The scores of the real paths can be calculated by summing the launch scores and transfer scores, and the sum of the scores of all paths can be calculated by the dynamic programming method. The BI-LSTM-CRF model is obtained by splicing the BI-LSTM and CRF through the fully connected layer.

Precision, recall, F1 value and accuracy are used as evaluation criteria for Q&A accuracy for the knowledge graph based intelligent educational system proposed in this paper. The experimental parameters for entity recognition in this paper are shown in Table 1.

In this paper, we use the entity recognition dataset based on the NLPCC-ICCPOL 2018 dataset and the pre-processed basic education dataset of the University of T. 1)

Experimental results based on NLPCC-ICCPOL 2018 dataset

This text recognition model increases the number of iterations in the training process, the loss rate changes, decreasing, to the end of the stability to maintain a certain degree of fluctuation, and then basically immobile, the loss rate changes as shown in Figure 2.

As shown in the figure, the horizontal coordinate indicates the number of times the entity recognition model of this paper is verified on the validation set data during the iteration process, and the vertical coordinate indicates the loss rate of the validation set output at the time of validation during the training process of the model, as can be seen from the image, the model has been trained continuously, the loss rate has been towards the decreasing trend, and finally tends to fluctuate to a stable value, which can be seen that the model has been trained.

Figure 2.

The changes of loss rate

The F1 value of the entity recognition model in this paper changes with the number of training times during the training process, as shown in Figure 3. The horizontal coordinate epochs in Fig. 3 represents the number of training times, and the vertical coordinate represents the change of the F1 value of the model during the training process, the figure shows that the model with the increase in the number of training times, the F1 continues to rise, and then stabilized and a little decline, it can be seen that in the training of the seventh time, the model trained the highest F1 value, the F1 value reached 98.67%.

Figure 3.

The F1 value of training

The results of different model tests are shown in Table 2. This paper proposes that the entity recognition model in this paper is compared with CRF, Bi_LSMT, Bi_LSMT+CRF, and BERT models can be found to be higher than other deep learning models in terms of precision rate P, recall rate R, and F1 value.

Table 2 specifically lists the individual deep learning models, as well as this paper proposes this paper’s entity recognition model of the precision rate P, recall R and F1 value of the data for comparison, and through the form of bar charts, a more intuitive feeling of the individual deep learning models in the entity recognition of the data task of the comparison of the deep learning model is shown in Figure 4.

Figure 4.

The comparison of the deep learning model

In Figure 4, the horizontal and vertical scales represent each deep learning model, and the vertical coordinate represents the value of accuracy, the first of each set of data represents the precision rate P, the second represents the recall rate R, and the third represents the value of F1, and it can be found that the three values of this paper’s model are higher than those of other deep learning models.

2)

Experimental results based on the basic education dataset of the University of T

Figure 5 shows the loss rate change. The entity recognition model in this paper increases the number of iterations in the training process, the loss rate changes, decreases, and at the end of the stable to maintain a certain degree of fluctuation, and then basically does not move. The horizontal coordinate indicates the number of times the entity recognition model of this paper is verified on the validation set data in the iteration process, and the vertical coordinate indicates the loss rate of the validation set output at the time of validation during the training process of the model, as can be seen from the image, the model has been trained continuously, the loss rate has been towards the decreasing trend, and finally tends to fluctuate to a stable value, which can be seen that the model has been completed training.

Figure 5.

The changes of loss rate

In this paper, the entity recognition model in the training process F1 changes with the number of training times, as shown in Figure 6. The horizontal coordinate epochs in Figure 6 represents the number of training times, and the vertical coordinate represents the change of the F1 value of the model during the training process, the figure shows that the model with the increase in the number of training times, the F1 value continues to rise, and then stabilized and a little decline, it can be seen that in the training of the 5th time, the model training the F1 value to the highest value of the F1 value reached 97.43%.

Figure 6.

The F1 value of training

In this paper, the user intention recognition task is converted into a question sentence classification task, and the user question sentences are categorized into five types: factual, statistical, whether, list, and method using this paper’s algorithm. The prediction results of this paper’s classifier for the five types of interrogative sentences are shown in Table 3.

By analyzing Table 3, it can be found that this paper’s algorithm has a good performance on the dataset, and the F1 scores of each type of questioning are around 90.0%, which indicates that the model has a good classification ability. Among them, the statistical type of questioning has the best classification effect, with a precision rate of 92.63%, a recall rate of 93.48%, and an F1 score of 93.05%. The F1 score of statistical type of question is slightly higher than the other four types, which indicates that the features of statistical type of question are more obvious compared to the other types, and the model has a better ability to recognize and capture the features of this type of question. The F1 scores of methodological question types are slightly lower, which may be due to the small number of samples of methodological question types included in the test set or the features of methodological question types are not obvious. In addition, the difference in F1 scores for the five question types is not very large, and the precision and recall rates are also relatively balanced, indicating that the model’s ability to classify each type of question is relatively balanced, and the overall classification effect is able to provide support for the task of recognizing user intent.

In addition, Support Vector Machine and K-Nearest Neighbor are selected as comparison models to verify the effectiveness of this paper’s algorithm, and the experimental results of multiple models are shown in Figure 7. It can be seen that the F1 scores of Support Vector Machine, K-Nearest Neighbor and this paper’s algorithm are all above 85.0%, in which the precision rate, recall rate and F1 score of this paper’s algorithm are higher than the other two algorithms. The precision rate of support vector machine is slightly higher than that of K-nearest neighbor, but its recall rate is much lower than that of K-nearest neighbor, indicating that the misclassification rate of support vector machine is higher than that of K-nearest neighbor.

Figure 7.

The comparison of the effects of multiple models

Table 4 shows the experimental comparison results of multiple models. Analyzing the experimental results in combination with the evaluation indexes shows that under the same dataset and feature selection, the precision rate, recall rate and F1 score of the support vector machine algorithm are lower, indicating that the model performs in general in this experiment, which is due to the smaller dataset. The K-nearest neighbor algorithm achieves a recall rate of 89.38%, but the precision rate and the F1 score are lower, which suggests that the algorithm may have over fitting phenomenon.

The algorithm in this paper outperforms the other two algorithms in this sentence classification task, and its precision rate, recall rate and F1 score are higher than those of the support vector machine and K-nearest neighbor by 4.18%, 6.41%, 5.3%, and 4.7%, 2.84%, 3.79%, respectively, which indicates that the algorithm in this paper has a high accuracy and generalization ability in this sentence classification task, and the classification effect is more satisfactory than that of the support vector machine and K-nearest neighbor. This indicates that compared with Support Vector Machine and K-Nearest Neighbor, the algorithm in this paper has higher accuracy and generalization ability on this question classification task, and the classification effect is more satisfactory. In addition, this paper compares the training time required by the three algorithms, because the K-nearest neighbor algorithm needs to calculate the distance between each test sample and the training sample, its training time is the longest, and the support vector machine algorithm and this paper’s algorithm require a relatively short training time.

In order to verify the practical effectiveness of this system, testing of the Q&A function is required. In this paper, 1500 question sentences suitable for the answer range of this Q&A system are selected for testing in the Chinese Learning Intentions dataset CMID, and this paper tests the practical use value of this medical Q&A system based on whether the above questions are correctly answered or not. The test mainly examines the following three indicators: 1)

Whether the test question was answered correctly.

Each time, 120 randomly selected questions from the dataset were tested on the Q&A system, and the results were compared manually for a total of 10 experiments, the results of which are shown in Table 5.

In Table 5, the accuracy rate indicates the number of correct answers as a percentage of the number of questions asked, and the precision rate indicates the number of correct answers as a percentage of the number of answers returned. In the final 10 test results obtained, an average of 101.2 answers were returned, of which an average of 90.7 were correct. In the event that a query was encountered that failed to parse successfully, the result returned was a pre-set failure response template, “I’m sorry, I can’t understand your question.”

All things considered, the Q&A system got a 75.6% correct rate, which is of some use. Analyzing the above test results, the Q&A system has the following problems: 1)

The data in the Chinese knowledge graph is not perfect enough, which contains only 36,000 Chinese entities in size, which may not be able to cover the entities in the user’s question, leading to the low accuracy of the Q&A system.