Research on the Method of Improving Accuracy of Martial Arts Movements Based on Time Series Analysis

With the vigorous development of computer vision and sensor technology, the application of human behavior recognition technology has become a popular research field in recent years. With the help of high-resolution video capture equipment, martial arts action recognition technology can accurately capture and record every subtle action of the martial artists, and through the rapid processing of the collected data by the computer, it can help professionals to analyze the performance of the martial artists, find out where the problems are, and provide personalized feedback and guidance for the martial artists, which helps to improve their sports skills. The traditional way of teaching martial arts is to judge purely with the human eye, based entirely on the experience of the instructor and the student's perception, which is difficult to provide more scientific and systematic support. Martial arts movement recognition technology can intuitively show martial arts practitioners the movement trajectory, strength and speed of each movement and other technical movement parameters, helping them to grasp the essentials of the movement more quickly, and they no longer rely on understanding the verbal expression of the instructor to learn martial arts movements. It can also be combined with virtual reality technology to provide an immersive training environment for martial arts learners to experience an intelligent martial arts teaching system, which greatly improves the efficiency of their mastery of movement essentials, and is also more likely to stimulate their motivation to learn and constantly break through to improve the level of martial arts.

Martial arts movement recognition technology can also be applied to assist judges in scoring. Martial arts movement judging relies solely on the subjective consciousness of the judges, which can easily lead to unfair scoring results. Martial arts movement judging scoring rules can be entered into the Martial arts movement recognition system, which can more accurately assess the movement normality of martial arts performers, score martial arts performers during competitions and training, and assist the judges, which can reduce the pressure on the referees, reduce the subjective error of manual scoring, and improve the fairness of the event.

The martial arts movement database established based on the martial arts recognition technology can permanently save more difficult martial arts movements, providing a wealth of material information for martial arts enthusiasts and researchers, and facilitating their in-depth research on the kinetic characteristics and movement laws of martial arts movements, so as to pass on the essence and connotation of martial arts skills. In addition, combined with modern technological means, after the analysis of martial arts movements, innovation and optimization can be carried out to inject new vitality into the development of martial arts skills, so that the spiritual connotation and unique style of traditional martial arts can shine with new brilliance in the modern society.

Therefore, the research of martial arts movement recognition technology brings a lot of convenience to the teaching of martial arts, provides a strong support for the research of martial arts movement, and helps to innovate and improve the martial arts movement, which is of great significance for the development and inheritance of martial arts.

The popularization of traditional martial arts in the younger generation is seriously insufficient, and its inheritance and development are facing greater challenges, how to apply modern science and technology to the teaching and inheritance of martial arts has become an important issue in front of us. Human motion recognition technology has been applied in many sports programs, in the Olympic swimming and figure skating programs, coaches can use this technology to monitor the daily training of athletes, to adjust and improve their irregular movements and improve the training effect; and also as an "electronic referee" has been applied to the In addition, it has also been applied as "electronic referee" in the competitions, which greatly improves the accuracy and fairness of the refereeing.

Compared with other sports, there are fewer cases of applying action recognition systems in martial arts programs. Chieh Caijie studied the application of Taiji Push Hands Interactive System, which is an applied research on the recognition technology of individual martial arts movements [1]. Han Zhiyang established an image recognition system for decomposed movements of martial arts, which is based on the key technology of contour edge image features of images [2]. Jiang Huabei explored the application of 3D motion capture technology and martial arts movement mechanics modeling analysis in the field of martial arts, and found that it has a broad application prospect and important practical value for martial arts teaching and training [3]. Three-dimensional motion capture technology and modeling analysis technology, which can accurately capture and record every subtle movement of the human body or objects, have shown great potential in the field of movies, games, and sports training, which makes the special effects of movies more realistic, and the movements of the game characters smoother and more natural, and provides strong support for the research of sports science. Applying it to martial arts education and research is an unprecedented impetus to promote the inheritance and development of martial arts culture.

Many scholars have also applied advanced artificial intelligence to the study of martial arts recognition system. Yajie Xu carried out the application of AI intelligent evaluation system to the movement evaluation of Martial arts intermediate examination program, and the similarity between the scoring results and the manual scoring was close to 90%, which provided practical experience and reference for the subsequent in-depth research and application [4]. Liu Yucong et al. applied the OpenPose algorithm to the recognition of traditional martial arts movements, and verified the feasibility of the system by comparing the joint angle data of a standard demonstrator and a test subject [5]. Kang Jiang proposed an Involution-based feature extraction network based on deep learning for human pose estimation research, and an action recognition scheme that can effectively recognize martial arts movements in videos [6]. Shuai Zhou used Kinect-based action recognition technology to establish a standard action recognition system for taijiquan and applied it to training and judging, which plays an important role in improving the quality of athletes' practice [7]. Zhao Chao et al. established an auxiliary algorithm for decision-making in martial arts set competitions by describing actions through the motion of skeletal joint points [8].

Many scholars have studied the recognition mode of human movement based on time series. Cheng Hui takes human movement recognition as the research object, designs a specific acceleration data acquisition platform, and carries out time series feature extraction and model training for the recognition of daily human movements, such as from lying to sitting up, from sitting to lying down, from sitting to standing, and from standing to sitting down, and the results show that the human movement recognition system can effectively distinguish the daily movements of human beings. [8] The results show that this human motion recognition system can effectively distinguish human daily actions. Chang Wei studied the human movement recognition system based on video sequences, using 3D skeleton coordinate features as the feature extraction method, and also using multi-feature fusion to improve the accuracy of feature recognition. The multidimensional time series modeling method uses a sparse representation algorithm for time series analysis and a matching error minimization strategy for behavioral classification, and the results confirm that the method is very effective in improving the judgment of human behavioral classification and can effectively improve the recognition correct rate. Wen Qu focuses on the recognition method of human action in video based on time series, feature extraction and representation of behavioral features in video, then apply the method of time series analysis for time series analysis, and use k-nearest neighbor classification with rejection for behavioral classification, so that the accuracy of human action recognition is greatly improved, and also has a very good recognition for the recognition of unknown action data.

With the rapid development of video data acquisition equipment, the study of human behavior recognition technology in video has always been favored by scholars, however, the complexity and difficulty of action recognition in video has brought great challenges to the design of recognition technology. Due to the complex structure of the human body itself, and has differences in appearance, in the process of movement, may be due to the different swing of the limbs, will produce different actions, and height, gender, sports habits and other differences in the existence of the behavior recognition has increased the difficulty. Chinese Martial arts is profound, and Martial arts movements are even more complex, with huge differences in various footwork, stances, maneuvers and kicks. There are great similarities between different movements, and the composition of movements is even more diverse, making identification difficult. Moreover, the changes and complexity of the environment, such as light, also have a great deal of interference in the recognition of video movements, and similar movements will be judged differently due to different positions and viewpoints, so the complexity of the movement recognition task is much higher than that of the human body posture estimation task.

In this paper, the contour map is selected in feature extraction to represent the features of martial arts action, the feature representation of martial arts action is represented by the feature vector of time series, and finally k-nearest neighbor method is used for classification. The framework diagram of the time-series based Martial arts action recognition system is shown in Figure 2, which is roughly divided into four modules according to the processing function of each part: feature extraction module, feature representation, behavior modeling and action classification. The data during processing are in two spaces, image and time series, respectively, and the system collects and processes the data to gradually convert the image space to the time series data space. By extracting the statistical features of the time series, the laws in the time series data of martial arts movements are identified and used in the classification process. The whole system is divided into standard movement recognition phase and practice test phase. The standard action recognition phase is to collect the standard action demonstrated by the standard demonstrator for feature extraction and feature representation. The practice test phase collects the trainer's martial arts movements, performs feature extraction and representation, and then classifies them using the k-nearest neighbor method, and finally compares the output results with the standard movements.

Figure 2.

Framework of Martial arts movement recognition system based on time series analysis

The feature extraction module is the part of the dotted box in Figure 3.3, the process of detecting the martial arts action from the video, obtaining the contour features and extracting the time series data from the contour sequence. The feature extraction module is the key to determine the final behavior, and effective feature extraction can improve the accuracy of the recognition results. The methods of human behavior recognition feature extraction can be divided into global features (e.g., spatiotemporal body, contour, optical flow, motion history map, etc.), local features (e.g., HOG, HOF, 3D SIFT, grid-based representation), and skeleton features. In this paper, spatio-temporal wheel temples are used to describe martial arts movements, and the KPCA method is used to compress the spatiotemporal contour map data with a large amount of data, which is finally converted into time series data.

Figure 3.

Schematic diagram of the process of representing the rotunda sequence

A silhouette expresses information about the edges and shape of a target, and is a direct description of the shape of a human body's movement, which clarifies the orientation of local details and the approximate scale of the image. By comparing silhouettes, it is possible to recognize objects or distinguish whether they are of the same type. Spatio-temporal contours can not only represent the image content, but also effectively retain the global information compared with the spatio-temporal points of interest that focus on local information, and when the background of the video is fixed, the extraction of contours is relatively simple and easy. As shown in the schematic diagram of contour representation in Figure 3 [9], each contour image is a binary image of rixci, and the data in the vector r_i are sequentially the pixel values of the first row, the second row, ... to the r_i th row of pixel values in the contour r_i.

Feature Extraction The dimension of the preliminarily obtained features is very high, and the computational cost of direct classification and recognition is too high, so the dimensionality reduction process is usually performed first. Kernel function based 1 ∅:RN→ F,x1→X \[\varnothing :{{\text{R}}^{\text{N}}}\to ~\text{F},{{\text{x}}_{1}}\to \text{X}\] principal component analysis KPCA, a linear dimensionality reduction method, maps the original data to a high dimensional feature space F through the mapping function Φ, and then performs principal component analysis on the feature space F. The correlation is as in Eq:

After substitution derivation, if the projected principal component Vj (j=1,...,M) on the eigenvectors in F is finally computed, and let x be a test sample point, then its transformation on 2 (Vj)Tϕ(x)=∑i=1Mαijϕ(xi)Tϕ(x)=∑i=1mαiK(xi,x) \[\begin{matrix} {{({{\text{V}}_{j}})}^{\text{T}}}\phi (\text{x})=\sum _{\text{i}=1}^{\text{M}}\alpha _{\text{i}}^{\text{j}}\phi {{({{\text{x}}_{\text{i}}})}^{\text{T}}}\phi (\text{x)} \\ =\sum _{\text{i=1}}^{\text{m}}{{\alpha }_{i}}\text{K(}{{\text{x}}_{\text{i}}},\text{x)} \\ \end{matrix}\] F is Φ(x),which can be derived as follows, where K(xi ,x) is called the kernel function, and the choice of kernel function has to satisfy the Mercy's theorem.

The feature representation module is the part that goes from the interval sequence data to the feature vectors, where the time series data are analyzed to get various features that represent the martial arts movements and to define the representation methods. The methods of deterministic analysis of time series are trend and seasonality analysis, self-similarity analysis and periodicity analysis, so as to obtain the pattern of changes of martial arts movements over time. In this paper, the cycle estimation method is applied to analyze the cycle of the extracted time series of the wheelhouse aspect ratio as one of the features for action recognition later. The features are extracted from the process of transforming the video into time series to find the patterns of different time series of the same martial arts movements to discriminate the new martial arts movements. The difference between adjacent maxima in the autocorrelation sequence is the period value, and the aspect ratio sequence was smoothed before performing the period analysis. The period characteristics are denoted by X^T, and for the time series TS, its period characteristics are represented by the formula: 3 XiT=(∑k=1nTSkT+i)/n,i=1,2,……, T \[\text{X}_{\text{i}}^{\text{T}}=\left( \sum _{\text{k=1}}^{\text{n}}\text{T}{{\text{S}}_{\text{kT}+\text{i}}} \right)/\text{n},\text{i}=1,2,\ldots \ldots ,~\text{T}\]

In order to reflect information about changes in gestures over time, behavioral modeling prior to behavioral classification can more accurately distinguish between different classes of expressions. Methods used for behavior modeling include Hidden Markov Models, Linear Dynamic Systems, and Recurrent Neural Networks, which are in the process of development. In this paper, we adopt a multidimensional time series modeling method based on sparse representation, focusing on the sequential relationship of time, which can effectively express the time information and improve the classification accuracy. The model variables are shown in Figure 4 [10]. The sparse coding algorithm for time series analysis is.

Figure 4.

Descriptive diagram of sparse coding model variables

Where f_ak is the error factor for the accuracy calculation,wt is the number of information received by the system at time t, a_p,t is the number of information to be processed at time t for large motion, b_q,t is the number of requests received at time with small motion at time t, N_c is the number of calculated cores for information collection, and v is the average service efficiency of the data server.

The column vectors of the input matrix Y are the feature vectors of the corresponding frames, t is the number of video frames; the dictionary D is divided into several groups of sub-dictionaries, s is the number of sub-dictionaries, and each group of sub-dictionaries consists of several feature basis vectors (column vectors); the sparse coefficient matrix α is a group of sub-coefficient vectors of the same color, according to the grouping of the dictionary D.

The action classification module is from feature vectors to using k-nearest neighbor classification to get the classification result. Due to the inherent complexity of the structure of the human body and the diversity of martial arts movements, it is impossible to capture every action.

In this paper, a semi-automated action recognition is established by using k-nearest neighbor classification. k-nearest neighbor classification is a frequently used classification method in action recognition, which is to find the k nearest samples to the new sample in the data set, and then judge the class of the new sample according to the class that the k samples belong to. The formula for the simple k-nearest neighbor algorithm is as follows, where if a=b then δ(a,b)=1, otherwise δ(a,b)=0.

To further optimize the complexity of the action classification simulated by the k-approach method, we designed the optimization based on the above model 10 Et+1=Et+εcPc−εdPd \[{{\text{E}}_{\text{t}+1}}={{\text{E}}_{\text{t}}}+{{\varepsilon }_{\text{c}}}{{\text{P}}_{\text{c}}}-{{\varepsilon }_{\text{d}}}{{\text{P}}_{\text{d}}}\] 11 Emin≤Et≤Emax \[{{\text{E}}_{\min }}\le {{\text{E}}_{\text{t}}}\le {{\text{E}}_{\max }}\] 12 0≤Pc≤ZcPc,max \[0\le {{\text{P}}_{\text{c}}}\le {{\text{Z}}_{\text{c}}}{{\text{P}}_{\text{c},\max }}\] 13 0≤Pd≤ZdPd,max \[0\le {{\text{P}}_{\text{d}}}\le {{\text{Z}}_{\text{d}}}{{\text{P}}_{\text{d},\max }}\] 14 Zc+Zd≤1 \[{{\text{Z}}_{\text{c}}}+{{\text{Z}}_{\text{d}}}\le 1\] 15 Zd,Zc∈{0,1} \[{{\text{Z}}_{\text{d}}},{{\text{Z}}_{\text{c}}}\in \{0,1\}\] E_t+1, E_t represent different action moments, P_c and P_d represent the behavior patterns of different actions at that time, ε_c and ε_d represent the amplitude of the action respectively; E_min and E_max represents the minimum amplitude and maximum amplitude of the action represents Z_c and Z_d, representthe state factor of the human movement process, P_c,max and P_d,max, the maximum amplitude corresponding to different behavior patterns.Simple k-nearest neighbor algorithm has a lot of shortcomings, and its improvement algorithms are: distance-weighted nearest neighbor algorithm, cross-validation, and nearest neighbor algorithm with rejection, which can improve the recognition rate of the system. In this paper, we finally analyze the tester's martial arts movements by using k- nearest neighbor with rejection, which effectively detects and judges the query points that do not belong to the known categories by setting a radius R and ignoring the points whose distance is larger than R. The system can be used to identify the points that do not belong to the known categories.

In this study, the lunge punch, kick punch, and horse stance punch movements in the Junior Chain Fist Set are taken as the research objects. The research indexes were selected in the same side of the horse stance punch, lunge punch, bullet kick punch 3 movements, the specific indexes are defined in Table 1. 5 standard demonstrators were selected for each movement to demonstrate the standard movement for the feature extraction, and then the movements of 5 testers were collected for the feature extraction.

In order to verify the effectiveness of the time series analysis method used in this paper, a comparison of the correct rate of recognition of martial arts movements was done by using 2 recognition methods, namely, features based on time series analysis and contour map based on static features, as shown in Table 2. The results show that for the recognition rate of Junior Chain Fist movements, the accuracy rate based on time series analysis is higher than that based on static feature contour map. When the difference between the movements is relatively small, the recognition accuracy of the static feature-based contour map decreases significantly, while the accuracy based on time series analysis is relatively high.

Comparison of the recognition rates of different juvenile chain punching movements demonstrated by five standard demonstrators, the results are shown in Figure 5, which shows that the recognition rate of the backside horse stance punch fluctuates less, indicating that the degree of standardization of this movement is relatively high among each demonstrator; in addition, there is a small difference in the recognition rate of each movement of demonstrator 5.

Figure 5.

Recognition rate of juvenile serial punching movements of 5 standard demonstrators

Comparison of the recognition rates of different juvenile chain punching movements demonstrated by five testers, the results are shown in Figure 6, from which it can be seen that the recognition rate of each tester's movements is generally lower than that of the standard demonstrator, and the fluctuation of different movements is relatively large; the fluctuation difference between the same martial arts movements is also large, among which the fluctuation of the backside horse stance punching punch is smaller, and is similar to that of the standard demonstrator, which shows that the present time-series analysis method has a better effect on the recognition of the backside horse stance Punching Fist, indicating that this time series analysis method is more effective in recognizing the Back Horse Stance Punching Fist. The recognition rate of each tester can reflect which martial arts movement of each tester needs to be improved, which is of reference significance for the improvement of the accuracy of martial arts movements.

Figure 6.

Recognition rate of juvenile chain punching movements for the 5 testers

As an excellent cultural heritage, traditional Chinese Martial arts have a broad mass base, and learning Martial arts can not only defend oneself but also help one's health. The practice of martial arts is a systematic process, including footwork, bodywork, maneuvers, footwork, eyesight, etc. The movements are complex and varied, and require persistent training by the practitioner. However, the current learning mode is often based on face-to-face instruction or watching videos, which makes it difficult for learners to learn and seriously hinders the inheritance and development of martial arts. Martial arts movement recognition system based on time series analysis, as a cutting-edge technology, can obtain all-round and multi-angle data such as movement trajectory, speed, angle of martial arts movements, etc. Applied to martial arts teaching, it can provide personalized training plans according to the physical conditions and learning progress of the learners, and also allow the learners to find out the deficiencies in their movements, and improve the accuracy of each martial arts movement, which not only helps to strengthen the The training effect of the students, but also will be conducive to the students to intuitively feel the essence of martial arts movements, can promote the continuous development of martial arts skills[11-12]. Applied to the study of stylistic features and technical characteristics of martial arts schools, it will provide development potential for the innovation of martial arts movements. In this paper, the martial arts movement recognition system established based on time series analysis has achieved good results in recognizing the movements of Junior Chain Fist, and there is a big difference between the standard movement demonstrator and the tester, which can reflect which martial arts movements of the tester need to be improved, and it can play the role of auxiliary judging of martial arts movements, help to judge whether the movements of the practitioner are standardized or not, which is conducive to the enhancement of the precision of the martial arts movements. Martial arts movement recognition based on time series analysis is a new exploration and innovation of martial arts teaching mode, despite the challenges, we hope that through different optimization and improvement can be truly applied to practice, helping to promote the traditional culture of martial arts.