The construction of evaluation system of ideological and political education effect assisted by deep learning

March 30, 2014 for the first time clearly put forward the concept of “core literacy”, clearly pointed out that the core literacy that is “students should have to adapt to the lifelong development and social development needs of the necessary character and key competencies” [20]. And the cultivation and enhancement of core literacy is placed in the overall deepening of curriculum reform, the implementation of the fundamental task of moral education.

Necessary character and key competencies point to the foundation of human behavior and work, namely, non-intellectual factors and intellectual factors, both of which are a kind of human nature as the subject of the force, respectively, for the scientific and humanistic dimensions of the quality. The two have both relative independence and intrinsic relevance. Relative independence is reflected in the fact that, in addition to the difference in connotation between the two, the necessary character focuses on the spirituality and morality of human beings and emphasizes altruism, while the key ability focuses on the initiative and creativity of human beings and emphasizes internalization, and there is also a difference between the two in the formation of the mechanism. The intrinsic relevance is demonstrated by the fact that they reinforce each other in their own formation, and the interaction and integration between them are also highlighted in the formation of core qualities.

In terms of the actual performance of an individual as defined by the connotation of core literacy, core literacy refers to his/her exposure to real and complex life situations with uncertainty. The ability to synthesize and apply structured disciplinary knowledge and skills, as well as disciplinary concepts, thinking styles, and investigative abilities generated through appropriate learning modes. And the ability to utilize their own internal drive to perceive and experience in real-life situations, and the comprehensive quality of complex problems from clarification to solution.

No matter what kind of teaching and learning activities, it is necessary to deal with the relationship between its internal elements, i.e., the elements of educators, learners, and the medium of education, which in the context of school education corresponds to the teacher, students, and knowledge, etc. The following five points are the decomposition of the elemental relationship processing of deep learning, which is an important characteristic of its important characteristics as well as an important criterion of whether it occurs objectively or not [21]. 1)

Association and structure, mutual transformation of experience and knowledge

Association and structure are both learning styles and learning contents, and have a dual nature. As forms of learning styles, association and structure deal with the intertransformation of human knowledge and students’ individual experiences.

Ideological and political education is briefly summarized as the cognition of cognition, and learners who master metacognitive knowledge and strategies can carry out self-monitored regulation of their own thinking, and its relationship with deep learning is mutually reinforcing. On the one hand, deep learning is both a concept and a learning strategy. Learners can gradually develop their metacognitive ability through the construction of knowledge, perception of context, and problem solving in the process of deep learning. [22]. Both are learning processes under the active control of the learner, which ultimately leads to the solution of the problem. On the other hand, the detection and regulation of thinking contained in ideological and political education and strategy can better guide the learners to actively experience the situation, perceive the problem, construct knowledge and solve the problem in the process of deep learning [23].

Corresponding to the deep learning process of context creation, knowledge construction, problem solving, and reflective evaluation, the above four cognitive theories are interrelated and have their own focus. The logical relationship between deep learning and the aid of ideological and political education is shown in Figure 1.

Figure 1.

The logical relationship of deep learning to assist ideological and political education

Core literacy and deep learning are the goals and methods of ideological and political education, i.e., the cultivation of core literacy cannot be separated from the occurrence of students’ deep learning, and the occurrence of deep learning is the practical direction and necessary conditions for the cultivation of core literacy. At the same time, adherence to the core literacy orientation is a prerequisite for the occurrence of in-depth learning, ideological and political education must adhere to the goal orientation of the core literacy, with the cultivation of the authenticity of students’ core literacy as the ultimate goal.

This study conducted a survey on the current situation of the effectiveness of ideological and political education in colleges and universities during the period of September 2023-March 2024 for college students at University D. A total of 2816 college students participated in the survey on the effectiveness of ideological and political education involved in this study, and Table 1 presents the distribution of the number of survey respondents, which mainly includes the basic information of the respondents such as their gender, age, grade, political appearance, academic specialty, place of origin and ethnicity, and other basic information.

In order to effectively supplement the questionnaire survey data, we need to compensate for the inadequacy of the standardized questionnaire survey in the thorough excavation of the real feelings and experiences of the survey sample. Based on the quantitative investigation of the real sample of the effectiveness of ideological and political education in colleges and universities, this study carries out in-depth interviews by applying qualitative research methods, taking into account the characteristics of different post specialties of college and university teachers and the specialties of students’ grades. Four types of groups were selected as the interview subjects of this study, including 30 current students, and 15 teachers, including teachers of ideological and political theory courses, teachers of professional courses, and counselors, for a total of 45 interviews.

NVivo software, as a qualitative research and analysis tool with coding as an important means, can visualize and systematize textual information. By importing a large amount of materials such as documents and pictures with complicated relationships into the system, the nodes can be modulated according to the research needs, and the key contents of the materials can be mined in depth relying on the relevant theories to realize the effect of three-level coding. In this study, the analysis is mainly divided into two steps based on the rooting theory:

The first step is the identification and determination of open coding, i.e., the collected information is imported into the software to generate a number of reference points, which are corrected, added, deleted and organized to be integrated into the initial category-based nodes.

The second step is the screening and generalization of the main axial and selective codes, i.e., under the guidance of the theoretical foundation, analyzing and classifying again, organizing and generalizing, constructing the tree structure (subordination between main categories), condensing the core categories from the systematic connection of each indicator, and finally forming the systematic evaluation indicators.

In this section, the textual information collected was coded with the help of NVivo12 software, and the main sources of the text were:

First, a total of 24 core journal documents that are closely related to the topic of this study and have a high citation rate, generating 481 reference points.

The second is a total of 7 policy documents related to ideological and political education collected from relevant websites, generating 121 reference points.

As a result, under the guidance of the CIPP evaluation model, the 602 reference points can be further screened and refined to produce the initial evaluation index system of the effectiveness of ideological and political education in colleges and universities for this study, and the specific steps are shown in Figure 2.

Figure 2.

Text data encoding process

After the end of open coding, it is necessary to continue processing the generated nodes and merge or delete any nodes that are duplicated or have similar concepts. The nodes with juxtaposition or subordination are categorized and summarized, and finally incorporated into the indicator system based on the four major elements of the CIPP evaluation model, to obtain the initial one, two or three-level indicator structure of the evaluation index system of the effectiveness of ideological and political education in colleges and universities as shown in Table 2. It consists of 4 level 1 indicators, namely, background evaluation of ideological and political education (A), input evaluation (B), implementation evaluation (C), and implementation effect evaluation (D). 10 level 2 indicators, and 26 level 3 indicators.

According to the theme of this study and the evaluation indexes initially derived, there are three rounds of expert consultation questionnaires in this part, and the expert group is mainly composed of ideological and political educators of University D, master and doctoral supervisors with outstanding achievements in ideological and political education research, and teachers with rich teaching experience, and the members of the expert group are shown in Table 3. The number of experts can not be too many or too few, too few will make the evaluation index system lack of credibility, too many will make it difficult to unify the opinions and face a large workload, so the final selection of 12 people as the expert group of the study consulting.

1)

The degree of expert enthusiasm

The degree of expert positivity is expressed by the total recovery rate of the questionnaire, and the size of its coefficient reflects the degree of expert interest in this evaluation index system, which is calculated by the formula: (1)K=mim

The rate of questionnaire comments is the ratio of changes proposed by experts in the questionnaire to the number of returned questionnaires, and is calculated by the following formula: (2)P=aia×100%

Where m and a indicate the total number of experts invited to participate in the construction of the evaluation indicator system. m_i indicates the number of experts who actually participated in the construction of the evaluation index system, and a_i indicates the number of experts who actually participated in the evaluation and provided written comments. K The larger the coefficient, the higher the degree of experts’ active participation in the study. The larger the P value, it can also fully explain the degree of experts’ concern.

It was calculated that the recovery rate of the first round of expert questionnaires was 91.67% (11), with a recovery rate of >70%, and the rate of opinions given was 58.33% (7), which indicates that the positive coefficients of each expert in the first round of questionnaires were good. The recovery rate of the second and third rounds of expert questionnaires was 83.33% and 75%, which were both greater than 70%, indicating that the experts in the second and third rounds of questionnaires had a high degree of positive coefficient. 2)

Expert authority coefficient

Generally speaking, the degree of expert authority is determined by the two factors of “the basis of expert’s judgment on the reasonableness of the indicator” and “expert’s familiarity with the issue”, which is expressed by the formula: (3)C=ci+cs2

Where, C indicates the total expert authority degree, C_i refers to the degree of influence of the judgment basis, and C_s is the degree of familiarity with the issue.

After calculation, the expert authority coefficient of the three rounds of the expert questionnaire C_r = 0.729 ≥ 0.7, indicating that the expert group has a high degree of authority. 3)

Concentration degree

Concentration degree E_i refers to the degree of concentration of experts’ opinions on the i th evaluation index, which is expressed by the formula: (4)Ei=1N∑jneijnij

Where N represents the total number of people consulted by experts; e_ij represents the value of (j=5,4,3,2,1) for evaluation indicator i with an importance level of j; and n_ij represents the number of experts who have determined that evaluation indicator i has an importance level of j. 4)

Dispersion

The degree of dispersion D_i indicates the degree of dispersion of experts’ opinions on the i th evaluation indicator, which is expressed by the formula: (5)Di=1N∑jnnij(eij−Ei)2

Where the larger the value of E_i, the more important the evaluation indicator proves to be. The smaller the value of D_i, the more centralized the opinion of the experts, and vice versa. When the results of the two cannot be consistent, it is necessary to look at the value of the coefficient of variation V_i, where the larger the value of E_i and the smaller the value of D_i, the smaller the value of V_i, which proves that the evaluation indicators are more important.

In particular:

When the coefficient of variation is V_i ≤ 0.1, it indicates a high degree of agreement among the expert groups.

When 0.1 ≤ V_i ≤ 0.2, it means that the harmonization of expert group opinions is within acceptable limits.

A coefficient of variation of V_i ≥ 0.2 indicates that the expert group disagrees and needs to be amended or deleted.

After three rounds of statistical analysis of the indicators in the expert questionnaire, as shown in Table 4, it can be seen from the calculation of the data that the coefficients of variation of all the items of the 4 level 1 indicators and 10 level 2 indicators are less than 0.2. It can be seen that the expert group agrees with the revised level 2 evaluation indicators with a high degree of consistency and high degree of importance, and therefore there is no need to add, delete, or modify the indicators again. Among the 26 tertiary indicators, the coefficients of variation of 24 indicators are less than 0.2, and the coefficients of variation of B12 and B31 are 0.243 and 0.205 respectively, which are greater than 0.2. According to the experts’ suggestions, the following adjustments are made:

First, the indicator “Teaching performance/B12” should be revised to “Disciplinary ecology B12”.

Second, delete the indicator “Team building/B31”, and replace the former B32 and B33 with B31 and B32.

After calculation, the consistency coefficient of the results of the three rounds of expert solicitation is 0.215, the significance coefficient P-value is 0.009, and the mean value of each indicator is greater than 4. The above test parameters indicate that the expert group has a high degree of acceptance of the indicator system after the three rounds of questionnaire revision.

After three rounds of Delphi research, the original indicators at all levels were fully verified and revised, and the structure of the effectiveness evaluation system of ideological and political education in colleges and universities was finally determined. It consists of 4 first-level indicators, 10 second-level indicators, and 25 third-level indicators.

The use of hierarchical analysis to give weight to the indicators requires the construction of two-by-two judgment matrices of the indicators on the basis of expert scoring, calculating the eigenvalues λ_max and eigenvectors W of the judgment matrix, and substituting them into Equation AW = λ_maxW to arrive at the weights of each indicator at each level of the order of importance of each indicator to an indicator at the upper level, a process known as hierarchical single sorting.

First regularize each column of the judgment matrix, i.e.: (7)bij=aij∑i=1naij(i,j=1,2,3…,n)

The judgment matrices after regularization of the columns are summed by rows, i.e.: (8)Vi=∑j=1nbij(i,j=1,2,3…,n)

The vector V=[V1,V2…Vn]T is then regularized and the resulting vector [w1,w2,w3…wn]T is the weight vector: (9)Wt=Vt∑i=1nVi(i=1,2,3…,n)

Finally, the maximum eigenvalue λ_max of the judgment matrix is calculated by substituting into the MATLAB software: (10)λmax=∑i=1n(AW)inWi(i=1,2,3…n)

In order to avoid self-contradiction when experts compare the indicators between two, and to enhance the rationality of the indicator weighting values, it is necessary to carry out a consistency test for the indicators, which is calculated by the formula: (11)CI=λmax−nn−1(n is the order of the judgment matrix)

Following this, the consistency ratio CR is calculated: (12)CR=CIRI

The average random consistency index (RI value) is shown in Table 6, and the hierarchical single-ranking results are considered to pass the consistency test when the random consistency ratio CR < 0.10.

In conjunction with the principles and methods of calculation of the hierarchical analysis method described above, a second round of questionnaires was distributed to enable the experts to score the importance of the indicators of the study, and Figure 3 shows the vector values of the judgment matrices of the first, second, and third level indicators after regularization.

Figure 3.

The vector value after the normalization of the judgment matrix of the evaluation index

Substituting MATLAB software to calculate the maximum eigenvalue of this judgment matrix is λ_max = 4.051, which is obtained according to the consistency index: CI=λmax−nn−1=4.051−44−1=0.017

Based on the listed RI values, the random consistency ratio is obtained: CR=CIRI=0.0170.84=0.0202

Therefore, it is considered that the distribution of the weights of the first-level indicators is reasonable and the experts’ opinions are relatively unified, and the results of the weights are shown in Table 7. The weights of the four first-level indicators are 0.2517, 0.1589, 0.2337 and 0.3557, respectively.

Similarly, according to the expert scoring results to construct the second and third level indicator judgment matrix of this research index system, and calculate its consistency indicators and ratios, and the integration results obtained are shown in Table 8, and the consistency ratios (CR value) of all the second and third level indicator judgment matrices are all less than 0.1, which passes the consistency test, and therefore the allocation of the weight coefficients is more reasonable.