An Analysis of the Effectiveness of Psychology Education in Higher Education Institutions Based on the ARCS Model

The following characteristics are required of those surveyed: 1)

They are participants in all aspects of psychology education and can provide opinions and suggestions from different perspectives of psychology education.

In this paper, 200 people, including leaders in charge of psychology courses, experts in the field of psychology education, staff of psychological counseling centers, front-line teachers of psychology education, and randomly selected students of various higher vocational colleges and universities in a certain province, were selected. Their numbers were 3 leaders in charge, 30 experts in the field of psychological health education, 35 counselors in the university psychological counseling center, 45 first-line teachers, and 87 students from various colleges and universities. Among them, experts in the field of psychology education refer to teachers who have the title of professor or associate professor and must be involved in at least two of the teaching of psychology education courses, psychological counseling, and management of psychology education in their personal work, counseling center staff refers to full-time staff of the Mental Health Education Center or the Counseling Center, and front-line teachers of psychology education refer to those who are mainly engaged in teaching in the classroom.

After the theoretical analysis, interview research and questionnaire survey in this paper, the content of indicators at all levels of the evaluation system of mental health education courses was finally derived, and its indicators and the content of the corresponding indicators are shown in Table 1: Table 1.

Index system content

In order to make the selection of indicator content more scientific, this paper invited 25 experts and professors to carry out a questionnaire survey on as well as indicators, secondary indicators and tertiary indicators respectively. The survey was distributed online, and 25 questionnaires were finally recovered, with a recovery rate of 100% and an effective rate of 100%. After the questionnaire survey was recovered, the scoring of each expert on the indicators was organized through Excel software and imported into SPSS26.0 software to calculate the mean, standard deviation and coefficient of variation of each indicator. Among them, the consistency coefficient is to determine whether the evaluation opinions of experts on each indicator have a high degree of consistency. Indicator expressions in the table of statistical analysis results in this paper are replaced by using their numbers.

Table 2 shows the results of the collation, statistics and analysis of the data of the first-level indicators. Among them, the average of the first-level indicators are all ≥3.5 points, and the coefficient of variation is <0.25, which can be seen that the experts have a high degree of recognition of the first-level indicators in the questionnaire.

Table 3 shows the results of the collation, statistics and analysis of the data of the secondary indicators. Among them, the average of the secondary indicators is ≥3.5 points, and the coefficient of variation is <0.25, which shows that the experts have a high degree of recognition of the primary indicators in the questionnaire, indicating that the 14 secondary indicators meet the requirements of the standard, and therefore all of them are retained. However, the coefficients of variation of the two secondary indicators “A3 and D1” are larger than the coefficients of variation of the other secondary indicators, which are above 0.1000, indicating that there are some differences in the evaluation opinions of the 25 experts on these two secondary indicators. Although there are some differences, these two indicators meet the requirements of the standard and are therefore retained.

Table 4 shows the results of the collation, statistics and analysis of the third-level indicator data. Among them, the mean score of all three-level indicators is ≥3.5, and the coefficient of variation is <0.25, which shows that the experts have a high degree of acceptance of the three-level indicators in the questionnaire, indicating that the 29 three-level indicators meet the requirements of the standard, and therefore all of them are retained. However, the coefficients of variation of the seven tertiary indicators “A31, A41, A42, B22, B23, D21, E31” are larger than those of the other tertiary indicators, which are all above 0.1600, indicating that there are some differences in the evaluation opinions of the 25 experts on these seven tertiary indicators. Although there are some differences, the seven indicators meet the requirements of the standard and are therefore retained.

In summary, each indicator in the evaluation system of this paper is in line with the standard requirements, but the coefficient of variation of some indicators is larger, which is due to the fact that there is a subjective choice of experts in the process of scoring, and each expert has different views and insights on each indicator, so there are some differences. However, from the overall data, the average score of each index is above 3.00, which has met the standard requirements, and the 25 experts in the questionnaire survey have no modification opinions. It shows that the evaluation system of this paper is reasonable in the assessment of psychology courses. Next, the evaluation system of this paper is used to score and analyze the psychology courses in actual institutions.

In order to derive the weights of the indicators of the evaluation system of the psychology teaching course so as to better analyze the evaluation results, this paper uses the sum-product method in the SPSSAU software to do further calculations on the survey data of the above evaluation system.

Taking Expert 1 as an example, Table 5 shows the analysis results of Expert 1’s rating data on the first-level indicators in SPSSAU software.

As in Table 6, expert 1 analyzed the evaluation indexes two by two, and through calculation, it was concluded that in the evaluation system of psychology teaching course, the weight of teacher and process was relatively large, each accounting for 0.3, that is, expert 1 thought that the level of teacher and the teaching effect were the most important. And the CR value is less than 0.1, then the judgment matrix satisfies the consistency test.

The 25 experts’ questionnaire data are calculated and analyzed using the above methods, resulting in Table 7, which is the consistency test results of the 25 experts’ questionnaire data. The data show that the CR value determined by 25 experts in the evaluation system of science teaching course weights is less than 0.1, then the judgment matrix meets the consistency test, and the consistency test are passed, the calculation of its average value and normalized to derive the weight of the teacher and the process is relatively large, accounting for 0.301 and 0.351, respectively, and the background of learning, teaching materials, and the effectiveness of the learning account for 0.112, 0.125, respectively, 0.111.

Table 8 shows the specific results of the weights of the first, second and third level indicators, and it can be learned that the weights of the first level indicators (A) background evaluation, (B) teacher evaluation, (C) teaching materials evaluation, (D) process evaluation, and (E) effectiveness evaluation are 0.112, 0.301, 0.125, 0.351, and 0.111, respectively. In the process of teaching psychology, the learning background, teacher, teaching materials, process and effectiveness are mutually restrictive and dependent on each other. Learning background and teaching materials are preconditions, process and teachers are guarantees, and effectiveness is the test. Their weights are in the following order: effectiveness < background < teaching materials < teachers < process. Among the secondary indicators, (B2) teaching level, (D1) teaching methods, and (C2) content characteristics have high weights of 0.555, 0.654, and 0.654, respectively, which are above 0.550, indicating that in the process of psychology teaching, the level of teachers’ instruction, teaching methods, and attractiveness of classroom content are the important factors affecting the quality of psychology teaching courses. Among the three-level indicators, (B12) psychology professionals, (B21) obtaining better evaluation from teaching supervisors, and (D21) rationally utilizing psychological means such as psychological tests in the classroom have higher weights of 0.757, 0.738, and 0.742, respectively, which are above 0.700, indicating that psychology teaching with psychology professionals conducting lectures, obtaining supervisors’ recognition, and diversifying teaching means are the guarantee of high level course effect.

In order to test the effect and quality of the psychology teaching design based on the ARCS model constructed in this paper in actual teaching, the teaching design of this paper was first introduced into the teaching of psychology courses in a higher vocational college, and then the evaluation system of the psychology courses in this paper was made into a questionnaire, which was used to evaluate the effect of the psychology courses after using the teaching design of this paper. A total of 150 questionnaires were distributed and 147 questionnaires were recovered, with an effective rate of 98.00%. Among them, 35 were experts, administrators and teachers on campus and 115 were students. This questionnaire is in the form of a Likert scale, and the average of each tertiary indicator is calculated, and two decimals are retained to get the score of the tertiary indicators. The secondary indicator scores were multiplied by the weights of their individual tertiary indicator scores. The primary and secondary indicator scores were obtained in the same way, and the sum was obtained by weighting the respective weights.

Table 9 shows the average scores of the three-level indicators, and it can be seen that the scoring values of the 29 three-level indicators are all between 3.00 and 5.00. Among them, (A22) organizing psychological culture festival to popularize the idea of mental health, (A23) actively carrying out extracurricular psychology education activities, (A41) providing pre-service training for new teachers, and (C22) psychological theories with examples to explain the four indicators scored high, all above 4.00. The 5 indicators of (A21) beautiful environment for students to study and live, (A32) offering elective courses such as Psychology of Success, (B22) better evaluation given by teaching supervisors, (B31) producing high quality course PPTs, and (C21) comprehensive knowledge of psychology theories have low scores, all below 3.30, which indicates that these 5 areas are areas that need to be improved in the process of teaching psychology in this institution.

Table 10 shows the average scores of the secondary indicators. It can be seen that all 14 secondary indicators scored values between 3.00 and 4.50. Among them, (D2) the use of teaching tools, (E1) the benefit of students 2 secondary indicators scored high, both above 4.00. And the 3 secondary indicators of (A2) campus culture construction, (B3) teaching attitude, and (E3) generalizable type of curriculum construction scored low, all below 3.20.

Table 11 shows the average scores of the level 1 indicators. It can be seen that the five level 1 indicator score values are between 3.00 and 4.00. The (B) faculty evaluation score is the highest at 3.91, which shows the high academic level and academic rigor of the main psychology faculty at this institution. In contrast, (A) contextual evaluation scores were lower at 3.59, indicating that the psychology learning atmosphere needs to be improved at this institution.

With the comprehensive evaluation model, the final comprehensive score of psychology course education in this institution can be calculated as 3.87. By comparing the evaluation conclusions in which excellent corresponds to 4-5, good corresponds to 3-4, fair corresponds to 2-3, and poor corresponds to less than 2, the results of this paper come under the range of 3-4, which indicates that the quality of psychology course education in this institution is better and is evaluated as good.