Extraction and analysis of individual features of students’ ideological and political education based on big data clustering algorithm

Interval-scaled variables are continuous measures of a rough linear scale. The clustering results of this type of variable may be strongly influenced by the units. Generally the smaller the units of the variable, the larger the domain of values and the greater the impact on the clustering results. Data can generally be standardized to reduce the impact of variable units on clustering results.

In the above equation, p is a positive integer. When p = 1, the formula represents the Manhattan distance; when p = 2, the formula represents the Euclidean distance.

The above distance formula has the following properties: d(i,j) ≥ 0; d(i,j) = 0; d(i,j) = d(j,i); d(i,j) ≤ d(i,h) + d(h,j).

When it is necessary to differentiate the importance of variables, weights can be assigned to the variables with the following formula: (7) d(i,j)=w1| xi1−xji |2+w2| xi2−xj2 |2+…+wm| xin−xjn |2

Binary variables have states: 0 and 1. When the variable is present, the value is 0; otherwise the value is 1. The possible values of binary variables are shown in Table 1. Where p = a + b + c +d.

Binary variables are subdivided into symmetric and asymmetric binary variables.

A variable is symmetric binary if its two states have equal value and same weight. An example is the gender attribute. The dissimilarity formula for symmetric binary variables is: (8) d(i,j)=b+ca+b+c+d

An asymmetric binary variable is one in which the outputs of the two states of the variable are not of equal importance, e.g., positive and negative results of a disease test are generally valued in the case of getting the disease that has less chance of occurring, i.e., the positive case, and the result is coded as a 1. The formula for the degree of dissimilarity of an asymmetric binary variable is: (9) d(i,j)=b+ca+b+c+d

A categorical variable is similar to a binary variable, but it can take more than two state values. For example, a color variable may have five values: red, yellow, orange, green, and blue. Assuming that the number of states of a categorical variable is M, there are two methods for calculating the dissimilarity of categorical variables: (1)

Simple matching method (10) d(i,j)=p−mp where m is the number of variables that take the same value for object i and object j, and p is the number of all variables.

Ordinal variables are divided into discrete ordinal variables and continuous ordinal variables. Discrete ordinal variables are similar to categorical variables except that the ordering of the M states is meaningful. For example, faculty title ranks are categorized as assistant professor, instructor, associate professor, and full professor. Continuous ordinal variables are similar to interval scale variables, but they do not have units; for example, the ranking of gold, silver, and bronze medals in sports competitions.

Suppose that some ordinal-type variable f describing an object has M_f ordered states, defining these states as 1,2,⋯⋯, M_f. The degree of dissimilarity with respect to a variable f can be computed using the following method: (1)

Replace the value x_if of the f variable of the ith object with the corresponding ordinal r_if ∈ {1,⋯⋯, M_f};

Variables of this type approximately follow Equation Ae^Bt or Ae^−Bt. where A and B are positive constants and t generally represents time. There are three ways to calculate the dissimilarity of this type of variable as follows: (1)

Use the same method as for dealing with interval scaled variables;

In actual data processing, the data object to be processed is often described by a variety of different types of variables. For objects described by mixed types of variables, the degree of difference between them can be solved using Equation (11). The degree of dissimilarity d(i,f) between object i and object j is defined as: (11) d(i,j)=∑f=1pδij(f)dij(f)∑f=1pδij(f) Where: p is the number of variables in the object. Indicator δij(f)=0 when x_if or x_if is missing, or when x_if = x_jf = 0 and variable f is an asymmetric binary variable; otherwise indicator δij(f)=1 .

Depending on the type of variable f, its contribution dij(f) to the dissimilarity between i and j is calculated as follows: (1)

f is an interval scale variable: dij(f)=| xif−xjf |maxhxhf−minhxhf (h traverses all objects).

Clustering is an unsupervised learning method that aims to group data objects in a dataset into distinct groups or clusters in such a way that the data objects within a group are highly similar and the similarity between groups is low. Clustering algorithms usually group data without prior knowledge or labeling. As one of the classical machine learning algorithms, clustering algorithms are characterized by simple principles, high efficiency and practicality, and have high applicability in several application areas such as image segmentation, document analysis, feature learning and market segmentation.

Fuzzy clustering technique was proposed in the late 1960s and is an improvement of clustering technique. It was developed on the basis of the previous traditional distance-based clustering techniques, aiming to solve the problems of traditional clustering algorithms in dealing with complex datasets. Fuzzy clustering is a clustering based on the fuzzy division of objects, combined with the fuzzy measure of the object, and its basic concept is that the data points in the clustering space can belong to different clusters at the same time, and each data point has a proportional weight to indicate the degree of its belonging in each cluster. The fuzzy clustering results are expressed in the form of fuzzy matrices, which can be used to jointly decide which clusters each object belongs to and how much it belongs to according to subjective and objective forces, which improves the intuition of the fuzzy clustering results.

FCM is the best known fuzzy clustering method. With the hard clustering algorithm represented by K-means, each feature vector can be a member of a cluster, and in FCM feature vectors can belong to one or more clusters with different degrees of membership, constrained by a sum of membership degrees equal to one.

Let X = [X₁,X₂,⋯, X_n] ∈ ℝ^d≈n be a dataset consisting of n samples containing d features defined by the following matrix equation: (12) X=[ x11x12⋯x1nx21x22⋯x2n⋮⋮⋱⋮xd1xd2⋯xd1 ]

The FCM minimizes an objective function that takes into account the distance from the feature vector to the cluster center, and the following objective function needs to be minimized: (13) J(U,V)=∑i=1N∑j=1cuijmxi−cj2 where m is the number of clusters clustered, i,j is the class labeling, u_ij denotes the degree of affiliation of sample x₁ belonging to class j. i denotes the ith sample, x is a sample with d dimensional features. c_j(j = 1,2,⋯, k) is the center of the j cluster, which also has d dimensions. It is a Euclidean metric. In addition, the following assumptions are made: (14) ∑j=1cuij=1,0≤uij≤1 where i = 1,2,⋯, n, j = 1,2,⋯, k. In order to search for the solution of equation (13), the following results are obtained by the Lagrangian function calculation under the constraints (14): (15) cj=∑i=1N1ijmXi∑i=1Nuijm (16) uij=1∑k=1c(xi−cjxi−ck)2m−1 where i = 1,2,⋯, n, j = 1,2,⋯, k. The partition matrix U consists of the affiliation u_ij of the j th eigenvector to the first cluster. In any loop, the FCM computes C and U by Eq. (15) and Eq. (16). If u1j(t) is the value of u_ij computed after t iterations, the iterative process stops with the condition as: (17) U(2)−U(l−1)=max| uij(2)−uij(i−1) |≤ε

In the equation ε > 0 is a pre-assigned parameter which is considered as a specified level of accuracy. The FCM algorithm pre-assigns the number of clustering centers, the fuzzification parameter m, which allows for the continuous computation of the degree of affiliation U and the clustering centers C. The FCM algorithm can be summarized as follows:

Algorithm 1: Fuzzy C Mean Clustering Algorithm (FCM)

Input: dataset D, number of cluster centers k, fuzzification parameters m, threshold ε

Output: set of clustering center vectors U

Ideological and political education in higher vocational colleges and universities needs to adjust the existing teaching system and carry out teaching practices precisely through student portraits, so there is an urgent need to obtain student portrait labels from the data collected in schools. Establishing portrait labels is the key work of user profiling, and the label name is a semanticized short text to summarize the function or meaning of the label. Based on the logic of teaching Civics class in higher vocational colleges and universities, the labels are extracted from students’ basic situation data, academic level data, social practice data, and physical and mental health data.

Through the above data collection and label extraction, the Civics teaching management department of higher vocational colleges and universities can draw portraits of upcoming students according to the levels of grades, majors, and classes before the start of the first semester of the new academic year, so as to reasonably formulate the teaching plan and arrange the lecturers. Firstly, at the grade level, the student portraits will be combined with the academic level data labels to make the first round of division for the freshmen who have never taken university Civics classes and those who have already taken Civics classes; secondly, at the major level, the student portraits will be combined with the basic information and the academic level data labels to make the second round of division according to the background of the disciplines of arts, sciences, engineering, agriculture, and arts, as well as the overall performance profiles of the majors, including the willingness to learn Civics and the ability to learn the rest of the subjects. learning ability) for the second round of division; again, at the class level, the student portrait will combine the basic situation and academic level data labels to carry out the third round of division according to the middle and high school situation and the enthusiasm for independent learning; finally, at the student level, the student portrait will combine the basic situation, social practice and physical and mental health data labels to carry out the third round of division according to the students’ individual profiles (including freshmen and former freshmen, nature of hukou, age, sex grade) , professional recognition, social service awareness, self-management and sound personality for the fourth round of division. The Civics teaching management part can, according to the teaching arrangement, start from the first round of division, the label extraction and classification integration of the target group, so as to construct a student portrait in line with the actual teaching.

The sub-trait of Diligence and Diligence, Aggressiveness, is strongly correlated. Aggressiveness responds to expectations that reflect the pursuit of competence and success in work or study. From campus card spending data, diligence can be measured by the amount of time a student spends studying. While there is no explicit metric, the frequency of presence on their study areas can be calculated as a proxy. The most obvious study areas include libraries and academic buildings. Colleges and universities, for security reasons, will set up access to the library to ensure that no one can enter the library without a campus card. Therefore, in order for students to gain access to the library, they must swipe their cards. In addition, students borrow books inside the library through their campus cards. In the academic building, although there is no access control, students usually fetch water in the academic building when school is in session or during self-study. In the academic buildings of Chinese colleges and universities, the schools usually install card machines for the machines that provide boiled water, so that students have to swipe their cards when they get boiled water, thus saving water. These consumption records can be considered a reflection of the students’ studies in the building.

For this reason, this paper calculates the number of times each student appears in the academic building and the library and uses it to measure the diligence indicator. That is, the more times a student appears in the library and the academic building, the more diligent he is. The correlation between diligence and academic performance is shown in Fig. 1(a) and Fig. 1(b), where the effect of the mean has been removed, making it possible to compare between different students. As can be seen from the figures, diligence and achievement rank are inversely related, with a mean Spearman rank correlation coefficient as high as -0.381. This observation suggests that students’ efforts will always be rewarded with good academic performance. In addition, we find that the correlation varies across semesters. In particular, the correlation coefficients of -0.152 and -0.149 in the first semester are significantly lower than in other semesters where the correlation is more than -0.261 One possible reason is that the first semester grades may still be highly dependent on what was learned in high school. Another reason is that many universities set up a strict routine during freshman year, resulting in less variation in behavior across students. It is worth noting that this paper includes the number of times students go to the print room as an indicator of diligence because students often need to print a lot of instructional materials to review before exams (20 days). The Cramér’sV value was used for one of the sleep patterns, while the others were Spearman’s correlation coefficients, and the p-values of all the correlations were much smaller than 0.001.

Figure 1.

Correlation between academic achievement and diligence

Literature examining the relationship between sleep patterns and student achievement suggests that students with good sleep habits are likely to achieve good grades. In particular, wake-up time and bedtime have a significant impact on student achievement, and students who stay up late and sleep in are generally less successful.

In the dataset collected from the schools, there were no actual wake-up and bedtimes. However, students need to use their campus cards to turn on the water, enter and exit the dormitory or take a shower. Therefore, in this paper, the first time and the last time of swiping the card each day are used as the wake-up time and bedtime. Considering the variability of the first and last time of the day for different students, this paper does not use too specific time to represent the sleeping pattern. Therefore, the frequency of the first swipe and the last swipe of each student’s campus card corresponding to the hour is calculated, and the timestamp of the highest frequency is used to denote the sleeping pattern of each student. Corresponding to wake-up and sleep patterns, there are only a few discrete values to represent them, for example, wake-up patterns are mainly concentrated at 6:00, 7:00, 8:00, 9:00, and 10:00, while sleep patterns are mainly concentrated at 22:00, 23:00, 24:00, 1:00, and 2:00. The correlation between students’ sleep patterns and grades is shown in Fig. 2(a) and Fig. 2(b).

Figure 2.

The correlation between sleep duration and student achievement

In Figure 2(a), students who wake up later generally have worse grades. Since the first class usually starts at 8:00 a.m., students waking up at 6:00-7:00 a.m. and waking up at 7:00-8:00 a.m. are considered to be early risers, and those who often wake up during this time have the best grades. In Figure 2(b), we see that students who stay up late also have lower grades, which may be due to indulging in recreational activities such as games or novels. In this paper, we put the sleeping pattern characteristics into individual characteristics as well. Notice that the sleep pattern feature is uniquely hot, so we cannot calculate the Spearman correlation coefficient and choose the discretized Cramér’sV value between grades and sleep patterns instead, with a mean of 0.0677.

The students’ consumption behavior data extracted and counted according to the consumption behavior description index is shown in Table 2. In order to make the characteristics of students’ consumption behavior more accurate, the average monthly consumption amount of the extracted students’ consumption behavior data is the amount of students’ consumption in the restaurant, the peak monthly consumption amount is the total amount of students’ consumption in the restaurant and the supermarket, and the average monthly consumption frequency is the number of times that students consume in the restaurant.

Considering the actual situation of schools and college students, it can be seen that the average value of the overall average monthly consumption of students is not large, indicating that students do not spend a lot in the restaurant, and the price of the school restaurant is more favorable, but the average monthly consumption frequency of the overall students is not high, indicating that the college students’ diets are more irregular, and they generally do not eat breakfast.