Machine Learning Based Big Data Analytics for Education in Curriculum Reforms

Machine learning is a core branch of artificial intelligence, with its main theoretical foundation derived from statistics, computer science and applied mathematics, and is a series of algorithms and techniques that enable computer systems to use data to learn and improve themselves. It enables computers to learn patterns from data by constructing models and use such patterns to make predictions and decisions about unknown data[5].

The main processes of machine learning are data preprocessing, feature selection, model training and evaluation[6]. Firstly, the quality of data is ensured through data preprocessing, which mainly involves operations such as data cleaning, normalisation and denoising. Next, feature selection is used to identify the most important attributes in all data to reduce dimensionality and improve model performance. After feature selection, algorithms are used to learn from the data to achieve model training results and discover potential relationships between the data. Finally, model evaluation is achieved by applying specific metrics to measure the accuracy of model predictions and generalisation ability[7]. Supervised learning, unsupervised learning and reinforcement learning are the three main types of machine learning. Supervised learning is learning a model to classify or predict new data from a labelled training dataset. Unsupervised learning deals with unlabelled data and focuses on discovering structure and patterns in the data. Reinforcement learning, on the other hand, trains a model to make optimal decisions in a given environment through reward and punishment mechanisms.

In 2012, Big Data for Development: Challenges and Opportunities, published by the United Nations, stated that the era of big data has arrived and will have a profound impact on all areas of society[8]. In the context of the rapid development of Internet technology in the new era, all industries have begun to use big data to better determine their strategies and business directions. Big data is characterised by a huge amount of data, rapid data processing and a wide variety of data, and it can process various types of data at a faster speed. In the field of education, the application of big data covers a diverse range of data sources from school teacher and student information management systems, online learning platforms, and course management platforms. These data contain both structured and unstructured information, including not only detailed records of students “learning progress, but also information on teachers” teaching activities, as well as data on various learning behaviours generated in online learning environments. Educational big data can be analysed in terms of clickstream data at the micro level, text data at the meso level and institutional data at the macro level. Micro-level clickstream data are mainly learners in learning environments such as smart tutoring systems, Massive Open Online Courses (MOOC), simulations and games. The textual data at the meso-level are mainly the data generated by students when they perform writing activities in learning environments such as smart tutoring systems, online forums, and social media. Institutional data at the macro-augmented level are mainly student personnel data, enrolment data, campus services data, course schedules and course enrolment data, university major requirements and degree completion data, etc., as counted by individual educational institutions. By collecting and analysing these educational big data, hidden educational laws and trends can be obtained from them, thus providing a more scientific and accurate basis for educational decision-making.

At present, machine learning is widely used in education big data analysis. For example, students' academic achievement and performance can be better mined and predicted by deep learning regression and linear review models[9]. Using the Hidden Malt customer service model can classify students “academic performance levels, forming a compact learning performance trajectory, which can obtain the relationship between students” learning performance trajectory and their ultimate academic success[10]. In recent years, the rapid development of students “online teaching, their online learning information is recorded, the study uses a large amount of interactive data from Moodle and the performance characteristics of college students based on the model of multiple linear regression, cluster analysis and correlation analysis to predict students” performance, and analyse the important factors affecting students' performance[11]. With the continuous progress of machine learning technology, its application in the field of education will be more in-depth and extensive, providing powerful technical support for educational reform and innovation.

Rerrent Neural Network (RNN) is a deep learning model that is suitable for processing sequence data of arbitrary length and can capture the temporal dynamic features in the sequence[12]. Its main feature is its short-term memory capability, which can solve problems such as temporal analysis and speech recognition. The above recurrent neural network can be represented by formulas, where formula (3.1) is the calculation formula for the output layer and formula (3.2) is the calculation formula for the hidden layer: (3.1) ot=g(Vst) (3.2) st=f(Uxt+Wst−1)

V—the weight matrix of the output layer,

g(Vs_t)—activation function,

U—the weight matrix of the input x,

W—the weight matrix of the last value as an input to this one,

f(Ux_t + Ws_t-1)—activation function.

Since recurrent neural networks are not suitable for processing long sequences, LSTM (Long Short-Term Memory, LSTM) with long and short term processing is obtained by improving on this basis.

Where, formula (3.3) represents the input gate, which indicates how much of the input xt of the network at that moment is saved to the unit state st, formula (3.4) represents the forgetting gate, who determines how much of the unit state st-1 at the previous moment is retained to st at the current moment, and formulas (3.5) and (3.6) represent the output gates, and the thing expressed in formula (3.7) determines the unit state st output value. The current value of the storage unit is shown by equation (2.9).

Random Forest (RF), proposed by Leo Breiman and his team, is an algorithm that performs well in classification and prediction tasks. It improves the accuracy of predictions by not significantly increasing the computational burden. The algorithm can efficiently handle datasets with a large number of feature dimensions without the need for feature dimensionality reduction[13]. In constructing a random forest, the generation of each decision tree is based on randomised putative back sampling of the original dataset N, with n samples taken for training, and this process is applicable to all trees. When the input data has m feature dimensions, a constant a much smaller than m needs to be set to ensure that the most optimal features from the m features are selected for splitting in each decision tree construction process. Ultimately, the prediction result of the random forest is derived from the combined prediction results of these independent decision trees.

The K-Nearest Neighbors algorithm (KNN) is a distance-based supervised learning algorithm mainly applicable to classification and regression tasks. It makes predictions based on the distance between test samples and training samples, usually using Euclidean distance, Manhattan distance or Minkows distance[14]. Also, it is a nonlinear lazy learning algorithm that does not require assumptions about the distribution or form of the data, and instead of constructing a model in the training phase, it uses the entire training dataset in the prediction phase. In classification problems, the KNN algorithm determines the class of the test sample by calculating the distance between the test sample and each sample in the training set and selecting the K nearest neighbours, and then determines the class of the test sample based on the majority vote of these neighbours. In regression problems, the KNN algorithm predicts the output of the test sample as a weighted average or simple average of the output values of the K nearest neighbours.

Equations (3.9) and (3.10) are the Euclidean and Manhattan distance formulas, respectively, according to which the final K-value is determined after tuning the parameters through experiments. The choice of K-value is crucial to the performance of the KNN algorithm and affects the bias and variance of the algorithm. Therefore, choosing the appropriate K value usually requires methods such as cross-validation to determine.

The aim of this study is to construct a student behavioural portrait and apply the K-means clustering algorithm to divide the students into groups in order to gain a deeper understanding of the student population and to provide customised pedagogical support. The research methodology is outlined as follows: first, the number of divided student groups is determined, followed by randomly selecting K data points as the initial centroids for clustering. In the final step, each data sample is categorised into the group represented by its nearest clustering centre. At the same time, the position of each clustering centre is updated to the mean of all sample points within that group. The distance between the sample and the clustering centre is calculated as described in Equation 4.3. Through this process, groups of students with well-defined characteristics are formed.

In personalised learning recommendation, the similarity between students is first calculated and then personalised learning content is provided to students based on the similarity. The similarity between students is calculated using the cosine similarity measure. Assuming that there are two students u and v, and the feature vectors of their learning behaviours are xu and xv respectively, the formula for calculating the cosine similarity is shown in 4.4.

In the formula, · denotes the dot product of vectors and ||x_u||||x_v|| is the mode of the vector x. The closer the cosine similarity is to 1, the more similar the two students are.

After calculating the similarity between the students, personalised recommendations can be provided to the students based on the similarity. The formula for personalised recommendation is shown below: (4.5) rul^=∑v∈N(u)sin(u,v)·rvi∑v∈N(u)sin(u,v)

rul^ —Student u's predicted level of interest in course i,

N(u)—is the set of k students with the highest similarity to student u,

r_vi—User v's level of interest in course i.

On the basis of the above, Python language and its related development frameworks are used along with Hadoop big data processing tools for recommendation of personalised learning. At the same time, the correctness, stability, performance and user satisfaction of the system were evaluated. The results show that the recommendation of personalised learning paths passes the test and performs well, providing students with a good learning experience.