The Application of Artificial Intelligence Big Model Technology in Mental Health Education Work in Colleges and Universities

The development of smart conversation plugins for the mental health domain requires consideration of special needs to ensure that the plugins provide accurate, sensitive, and appropriate support when dealing with mental health-related conversations. In the mental health domain, the emotional state of the user plays an important role in the quality and effectiveness of the dialog. Conversation models need to be able to recognize and understand the user’s emotions and respond to them appropriately. Applying deep learning techniques to sentiment analysis and emotion generation helps the dialog model to acutely perceive and process the user’s emotional expressions. Protecting user privacy and information security in the mental health domain is critical. Conversational models need to ensure that users’ sensitive information is protected and comply with legal and ethical guidelines for privacy protection. The design of AI models and data processing methods should follow relevant privacy protection measures, such as desensitizing sensitive information and adopting secure data transmission and storage methods.

Rasal is an open source task-based system framework implemented in python with high flexibility and extensibility. The designers have divided Rasa into two parts, NLU and Rasa Core. Among them, RasaNLU is responsible for understanding user input and extracting valid intent and entity information from it. Rasa Core contains modules such as conversation state tracking, conversation strategy, and conversation generation, focusing on conversation management tasks.

The architecture of Rasa Dialogue System is shown in Fig. 1. In the Rasa Dialogue System framework, the input text message passes through the Natural Language Processing module, Dialogue State Tracking module, Dialogue Strategy module, and Dialogue Action module in turn, and finally generates the reply content. The natural language processing module transmits the captured intent and entity information to the dialog state tracking module. The dialog state tracking module records and updates the intent and entity information captured during the entire dialog process. The dialog strategy module selects appropriate dialog actions based on the dialog state information. Finally, the dialog action module generates replies and updates the dialog state. The Rasa framework hides the tedious pipeline interfacing and dialog state tracking processes, allowing developers to focus on building individual modules.

Figure 1.

Rasa’s Architecture Diagram

Building a complete Rasa project can be accomplished by writing just a few files, which further reduces the cost of learning for developers.

Based on the current major problems in the field of rephrasing generation, this paper proposes a novel rephrasing generation model, 4SPG, which is based on the mainstream encoder-decoder neural network structure for characterization learning of input utterances. Among other things, this paper proposes a Bidirectional Paraphrase Learning (BPL) framework. The comparison between one-way repetition learning and two-way repetition learning is shown in Fig. 2. Current repetition learning frameworks all adopt a one-way learning model, i.e., the original utterance is used as input to learn the corresponding target utterance. In contrast, the bidirectional retelling learning framework demonstrated in this method will simultaneously learn to generate the original utterance with the target utterance as input. Specifically, the encoder and decoder in this framework receive the same paraphrase pair as input, and subsequently the decoder generates the paraphrase results corresponding to the original and target utterances, respectively. Compared to the manual expansion of the training dataset on a one-way paraphrase learning framework to achieve reuse of the training data, the high extensibility of the BPL framework makes it possible for the model to learn a deeper relationship between the original utterance and the target.

Figure 2.

Unidirectional paraphrase learning and bi-directional paraphrase learning

Observing Fig. 2, based on the BPL framework, this paper proposes a semantic enhancer and a stylistic enhancer based on a self-supervised learning approach to achieve semantic accuracy and stylistic diversity enhancement for the encoder and decoder, respectively. Among them, the semantic enhancer focuses on the semantic distance between two features and guides the encoder to capture more precise semantic information, thus improving the accuracy of the paraphrase. Stylistic enhancers emphasize potentially changeable regions through an attentional mechanism and cause the decoder to learn through backpropagation to generate richer representations. In particular, these enhancers are designed as auxiliary loss functions and do not require eliciting additional labeling data.

Specifically, given the input word embeddings of the original sentence x_s and the input word embeddings of the target sentence x_t, the outputs h_es & h_et of the semantic encoder SE(x) can be computed as equations (1), (2): 1hes=SE(xs)${h_{es}} = SE\left( {{x_s}} \right)$ 2het=SE(xt)${h_{et}} = SE\left( {{x_t}} \right)$

The outputs h_dt & h_ds of sentence decoder DE(x) can be computed as equations (3), (4): 3hdt=DE(xt,hes)${h_{dt}} = DE\left( {{x_t},{h_{es}}} \right)$ 4hds=DE(xs,het)${h_{ds}} = DE\left( {{x_s},{h_{et}}} \right)$

Next, this method maps h_dt and h_ds to a reasonable spatial distribution by means of a linear layer LMHead with a nonlinear activation function softmax, and the results y_s and y_t can be described as Eqs. (5), (6): 5ys=softmax(LMHead(hds))${y_s} = soft\max \left( {LMHead\left( {{h_{ds}}} \right)} \right)$ 6yt=softmax(LMHead(hdt))${y_t} = soft\max \left( {LMHead\left( {{h_{dt}}} \right)} \right)$

Unlike previous work on retelling generation, the BPL framework allows for simultaneous learning from both the original and target inputs. Assuming that N is the total number of words in the corpus, the cross-entropy loss can be used to fit the decoder input to the decoder output as in equation (7): 7Lce=−∑i=1Nxtlogyt−∑j=1Nxslogys${\mathcal{L}_{ce}} = - \sum\limits_{i = 1}^N {{x_t}} \log {y_t} - \sum\limits_{j = 1}^N {{x_s}} \log {y_s}$

The outputs h_es and h_et of the semantic coder preserve the deep semantic features of the original utterance and the target utterance respectively, which have rich semantic information. Based on this, a self-supervised semantic enhancer is designed in this paper, i.e., h_es and h_et are guided to generate more accurate semantic features by two dropout operations with strength 0.1 and minimizing the cosine distance between the enhanced features using a Lsem${\mathcal{L}_{sem}}$ loss function during the training process. The mathematical description is shown in Eqs. (8), (9), and (10): 8hes′=dropout(dropout(hes))${h'_{es}} = dropout\left( {dropout\left( {{h_{es}}} \right)} \right)$ 9het′=dropout(dropout(het))${h'_{et}} = dropout\left( {dropout\left( {{h_{et}}} \right)} \right)$ 10Lsem=−hes′⋅het′|hes′|⋅|het′|${\mathcal{L}_{sem}} = - \frac{{{{h^\prime_{es}}} \cdot {{h^\prime_{et}}}}}{{\left| {{{h^\prime_{es}}}} \right| \cdot \left| {{{h^\prime_{et}}}} \right|}}$

Unlike semantic enhancers, textual style migration is usually reflected in subtle lexical changes. For this reason, this paper implements style enhancement by introducing attention between the original features generated by the decoder and the target representation. Specifically, the style enhancer computes the difference between the hidden features h_ds, h_dt and the corresponding attention a_ds, a_dt values, and subsequently freezes the highly relevant information and maximizes the cosine distance of the lowly relevant parts through a Lsty${\mathcal{L}_{sty}}$ loss function. This is mathematically expressed as Eqs. (11)-(15): 11ads=softmax(hdshTdt)${a_{ds}} = soft\max \left( {{h_{ds}}{h^T}_{dt}} \right)$ 12adt=softmax(hdthTds)${a_{dt}} = soft\max \left( {{h_{dt}}{h^T}_{ds}} \right)$ 13h˜ds=softmax(hds−ads)${\widetilde h_{ds}} = soft\max \left( {{h_{ds}} - {a_{ds}}} \right)$ 14h˜dt=softmax(hdt−adt)${\widetilde h_{dt}} = soft\max \left( {{h_{dt}} - {a_{dt}}} \right)$ 15Csty=−h˜es⋅h˜et|h˜es|⋅|h^et|${\mathcal{C}_{sty}} = - \frac{{{{\tilde h}_{es{\text{ }}}} \cdot {{\tilde h}_{et{\text{ }}}}}}{{\left| {{{\widetilde h}_{es}}} \right| \cdot \left| {{{\widehat h}_{et}}} \right|}}$

Throughout the model training process, this method uses a total of three loss functions to measure the model performance, including the cross individual loss Lce${\mathcal{L}_{ce}}$ and the auxiliary enhancement loss Lsem${\mathcal{L}_{sem}}$, Lsem${\mathcal{L}_{sem}}$. Its total loss function L$\mathcal{L}$ can be expressed as equation (16): 16L=Cce+λ1Lsem+λ2Lsty$\mathcal{L} = {\mathcal{C}_{ce{\text{ }}}} + {\lambda_1}{\mathcal{L}_{sem}} + {\lambda_2}{\mathcal{L}_{sty}}$

where λ₁ and λ₂ are the hyperparameters corresponding to the auxiliary enhancement losses Csem${\mathcal{C}_{sem}}$ and Lsem${\mathcal{L}_{sem}}$, respectively, and are set to 1.2 by default.

UniLM, Unified Pre-trained Language Model. From the name of this model, we can know that it can train various language models by using different Mask strategies with the same parameters through a Transformer model.

UniLM is mainly stacked with Transformer blocks. The UniLM model used in this paper consists of a stack of 12 Transformers, and the hidden layer of each Transformer contains 768 hidden nodes and 12 heads. The UniLM is consistent with the structure of BERT-BASE, so the parameters can be initialized using the already trained BERT-BASE. The chance of masking a participle is 15%, and of the masked participles, 80% are replaced with [MASK], 10% are randomly replaced with words from the lexicon, and the rest of the participles are not processed. Further, when masking participles, a different number of participles were covered each time. With an 80% chance of covering one word, there is also a 20% chance that 2-3 consecutive participles can be covered at the same time.

UniLM can be trained using three Mask approaches for unidirectional language models, bidirectional language models and Seq2Seq language models, respectively, and they differ fundamentally in that the range of information obtained from words in the same sequence varies. For example, the seq2Seq language model, UniLM utilizes BERT in generative tasks via the Mask matrix. Although the model only uses the encoder architecture, it can be used for the Seq2Seq task. The reason for this is as follows: the words in the input sequence are controlled by the Mask matrix, which gives the contextual information of the words, and although the words that are about to be predicted in the target sequence have access to the contextual information of the input sequence, they only have access to the left side of the target sequence. The empathic response generation technique proposed in this section is based on the Seq2Seq language model, and so the self-attentive Mask mechanism of the Seq2Seq language model is discussed below.

In the pre-training phase, the user-input statement or question and the consultant’s empathetic response are organized into the form of contextual clauses and imported into the model. If the masked participle of the Seq2Seq language model belongs to Segment1, i.e., the original text sequence, i.e., the statement or question input by the user, then the participle can only focus on the other participles of the sequence in which it is located, and it cannot focus on the participle that belongs to Segment2, i.e., the target sequence, i.e., the empathic response of the counselor. If the masked participle belongs to Segment2, it can not only focus on the participle belonging to Segment1, but also see the information of the left participle in the same sequence.

For example, there are two sentences: “My classmates hit me” and “It’s miserable”. Seq2Seq language model input build, “[CLS] classmate hit me [SEP] [MASK] miserable [SEP]”. M refers to the words that have been masked by the MASKSeq2Seq language model for only 15% (“really miserable”). The “hit” in the first sentence is in a state of self-attention, and only all the words in the first sentence can be seen, and the words in the next sentence cannot be seen. [MASK] can see all the words in the first sentence and the above information of the sentence in which it is located.

During decoding, if there is statement “X₁”, at t = 1, “[CLS]X₁[SEP]Y₁[MASK]” is the input sequence, and adding the “[MASK]”’s corresponding feature to the end of the sequence means to make a prediction for the next word. In the encoding phase, the left sequence “[CLS]X₁[SEP]” is known and can focus on the contextual information. “Y₁[MASK]” is the target sequence, i.e., the sequence we want to generate, which is in the decoding state and can see the information of the known sequence and part of the information on the left side of the target sequence. Thus the model utilizes the mask matrix to organically combine the encoder and decoder.

The samples were coded using the UniLM model. The first row of the matrix refers to the representation of [CLS] features, the second row is the feature representation of Xi, and so on. For decoding, the feature representation through [MASK] passes through a linear layer, then, the probability distribution of the words in the word list is obtained using Softmax, and the word generated by decoding is the one with the highest likelihood, and the cycle repeats, terminating when [SEP] is generated.

Specifically, firstly, the attentions distribution of the under-head position to the remaining positions is obtained by Multi-head-attention, which is controlled by the mask matrix to control the range of each position or word that can be attended to, and then, the feature vectors of the under-head position of the decoder are computed as in Eq. (17): 17At=Softmax((Wq*Xt)*(Wk*XInput)Tdk)+M${A_t} = Soft\max \left( {\frac{{\left( {{W_q}^*{X_t}} \right)^*{{\left( {{W_k}^*{X_{Input}}} \right)}^T}}}{{\sqrt {{d_k}} }}} \right) + M$

A₁ is the attention distribution of the generated word vectors at t time pair X_Input, X_t is the target vector at time t, X_Input is the contextual feature vector at t time, M is a mask matrix controlling the attention range of the word, d_t is the word vector dimensionality, and W_g, W_k, and W_v are the learning parameters, which are related as in equation (18): 18XOutput=At*Wv*XInput${X_{Output}} = {A_t}^*{W_v}^*{X_{Input}}$

X_Output represents the feature vectors output from t the temporal decoder, which are linearly transformed twice for X_Input and then using the Softmax function, the final word list distribution P_v is obtained see equation (19): 19Pv=Softmax(W′(W*XOutput+b)+b′)${P_v} = {\text{ }}Soft\max \left( {W^\prime \left( {W^*{X_{Output}} + b} \right) + b^\prime } \right)$

where W, W′, b, and b′ refer to the learnable parameters.

The Softmax function maps the vector of scores s to a probability distribution, which is defined in equation (20): 20Softmax(s)j=esi∑j=1nesj${\text{ }}Soft\max {(s)_j} = \frac{{{e^{{s_i}}}}}{{\sum\limits_{j = 1}^n {{e^{{s_j}}}} }}$

i is the output node number. s_i is the value of the i rd node. n is the number of output nodes and also the number of categories.

Further, the cross-entropy loss is computed for the result X_output, denoted as s, along with the original disambiguation s_t, which aims to optimize the parameters of the model. The cross-entropy function is defined as in equation (21): 21L=log(∑i=1nesi)−si$L = \log \left( {\sum\limits_{i = 1}^n {{e^{{s_i}}}} } \right) - {s_i}$

In order to improve the problems of inaccurate details of generated complex events and the absence of emotional keywords in the generated empathic responses mentioned at the beginning, in this paper, we introduce the generation probability P_g into the UniLM model, P_g represents the probability of the complex event information and emotional keywords contained in the generated responses in the vocabulary, and P_z = 1 − P_g represents the probability of the probability of the possible copying of the complex event information and the emotional keywords from the user-inputted statements or questions. Figure 3 shows the structure of the UniLM-Copy mechanism model, which contains the Copy mechanism.

Figure 3.

UniLM-Copy mechanism model structure

P_g is obtained from the following calculation, see equation (22): 22Pg=Sigmoid(W[Xt,XOutput,At]+b)${P_g} = Sigmoid\left( {W\left[ {{X_t},{X_{Output}},{A_t}} \right] + b} \right)$

W and b denote the learnable parameters. The final probability distribution can be obtained as in equation (23): 23P(w)=Pg*Pv(w)+Pc*At$P(w) = {P_g}^*{P_v}(w) + {P_c}^*{A_t}$

Where, w is not part of the word list, P_v(w) = 0, the predicted words are generated from the source sequence, i.e., the user’s statement or question. w does not belong to the source sequence, A_t = 0, the predicted words are generated from the word list. Copy mechanism copies emotional keywords and complex event information from the user’s statement or question to the generated empathic response, thus achieving the empathy purpose.

This chapter focuses on the study of empathic response generation based on the UniLM-Copy model, which firstly designs and analyzes the principles of the 4SPG algorithm of the retelling generation model, and then proposes a mental health model based on UniLM-Copy, which is able to incorporate the complex event details and the emotional keywords into the generated empathic responses using the Copy mechanism. The model generates empathic responses by analyzing the potential emotions of the contextual information of the user’s input statements and combining them with mental health knowledge, thus guiding the user’s positive mental state.

In order to evaluate the effect of model response generation in this paper, this subsection compares the effect of automatic and manual evaluation of response generation of this paper’s method with Vanilla prompting method and CoT prompting method, respectively.

In this section, the NQ dataset, the TQA dataset, and the WQ dataset are used for the comparative evaluation of different algorithms for linguistic querying and comprehension, where the NQ dataset is used for the information retrieval system and the question and answer task, the TQA dataset covers common sense questions and the corresponding answers in several domains, and the WQ dataset is used for the question and answer task.

Top-K accuracy is one of the key metrics to measure the retrieval effectiveness of the model, which is used to evaluate how often the model is able to find at least one relevant document inside the top K retrieval results it returns in all queries. Expand the performance comparison between this paper’s algorithm and other algorithms in the six-column dataset of TQA Top-100(X1), NQ Top-100(X2), NQ Top-20(X3), TQA Top-20(X4), WQ Top-100(X5), and WQ Top-20(X6).

Figure 4 shows the TOP-k accuracy data for all algorithms in the dataset, where S1 is the algorithm of this paper, S2 is the DPR algorithm, S3 is the ANCE algorithm, and S4 is the BM25 algorithm.

Figure 4.

Retrieval performance comparison

Among all the algorithms, the BM25 algorithm has the worst results on all the datasets, which also indicates the importance of deep learning in text retrieval. The DPR algorithm, as the founder of dense retrieval, still has a good performance in terms of metrics, but it still has no advantage over other dense retrieval models that are improved on DPR. The model in this paper, on the other hand, shows an advantage over the baseline model BM25 in all cases. In terms of Top-20 metrics, this paper’s model outperforms BM25 by 21.4% on the NQ dataset, 15.5% on TQA, and 20.8% on WQ. This shows that the model in this paper has a significant advantage over traditional algorithms in understanding natural language queries.

In order to clarify the feasibility of the model of this paper in the practical application of mental health education in colleges and universities, this section develops the overall effect test and feasibility test of the model in terms of the positive guiding effect of the model on students and the stable operation of the actual scenario, respectively. The overall effect test adopts the form of comparing similar algorithms, while the feasibility test is carried out in a multi-user scenario.

4.3.1

Overall effect

In this paper, a total of 500 students in five majors in the second year of a university were psychologically assessed, and the top 20 students with anxiety scores in each major were selected as the experimental subjects, and were divided into groups A, B, C, D, and E according to their majors. Among them, the students in group A used the conversation model based on Vanilla prompting method, the students in group B used the conversation model based on CoT prompting method, the students in group C used the conversation model based on DPR algorithm, the students in group D used the conversation model based on ANCE algorithm, and the students in group E used the model in this paper.

The test data before and after the experiment were analyzed, and the mean values were calculated using the test entries of items 2, 17, 23, 33, 39, 57, 72, 78, 80, and 86 of the SCL-90 scale about anxiety test.

Since there are some differences between students of different majors in terms of student population, professional learning background, education and teaching, and the anxiety levels of the students in the five selected sample groups are not the same, the mean value of the above test entries is presented to reflect the anxiety level of the group. The test results obtained after the pre-test and post-test are shown in Figure 5.

Figure 5.

Experimental test comparison

As can be seen from Figure 5, the anxiety level of each group decreased after using the model, which basically proves that the model can provide a certain degree of help for the mental health education of college students. Among them, the anxiety level of students in group E is the lowest after using the model, and the average score is only 15.9, which is 3.0 different from the average score of anxiety before the experiment.

Figure 6 shows a comparison of the distribution of the effect scores of the five models on students’ psychological guidance in the experiment, out of 5 points. The data proves that the model of this paper has the best performance on students’ psychological guidance effect among similar models, and its performance score is as high as 3.89.

Figure 6.

Guidance mode comparison