Research on Motion Capture Technology of Peking Opera Performing Arts and Its Practicality Exploration in Opera Physical Training

China’s traditional folk art has a long and colorful history, and many of its art forms still influence us today. Many of them still influence us today, such as paper-cutting, opera, New Year’s paintings, shadow puppets, embroidery and so on. Among them, Chinese folk opera has its unique artistic charm and its popularity is rare among other traditional folk art forms. Its popularity is rare among other folk traditional art forms. It can be seen in all places of our life. Folk opera has the ordinary characteristics of drama. And has its own unique aesthetic characteristics and methods of expression, both different from the drama to say, each with a singing-based, dance-based dance drama, but also a combination of opera, drama, dance, etc., has an extremely broad base of the masses. Most Chinese folk operas use singing, reciting, acting and playing as the basic means, through which actors perform stories, react to life and create the mood of the scene on the stage, and they widely synthesize many art forms such as poetry, literature, music, dance, drawing, sculpture, acrobatics, martial arts, etc., and have gradually become a comprehensive art form with a wide range and many titles. On the other hand, Chinese folk opera is a form of opera characterized by dialogues and movements, which is formed on the basis of the full development and fusion of various artistic components of literature, folk rap, music and dance.

As technical means of shooting contemporary film and television dramas, motion capture and digital animation techniques are widely used. However, due to the differences in artistic languages and performance modes, their application in traditional opera stages is still relatively rare. However, there are many mythological plays in Peking Opera, which involve a large number of martial arts actions and magical illusions, etc. If appropriate motion capture and digital animation technology is applied to these contents, it will greatly enhance the ornamental and visual expression of the plays, and arouse the resonance of the young people, thus prompting them to get in touch with and love the art of Peking Opera more. Combining motion capture and digital animation technology with Peking Opera movement training tries to break the limitation of stage space, enhance the stage performance of traditional opera, and interpret and disseminate the traditional art in a vivid and imaginative way. This is not only beneficial to the preservation of the art of Peking Opera, but also makes the ancient art conform to the aesthetics of the times and shine with the luster of the new era.

Motion capture technology is mainly used to record the movement information of the main joint parts of the human body through a variety of technical means, so as to achieve the recording and analysis of human movement information. Motion capture data can record the spatial position of joint parts, speed angle, and other data, as well as finger movements, facial expressions, and other data. At present, motion capture system is mainly divided into optical motion capture system, inertial motion capture system, mechanical motion capture system, etc., of which optical and inertial motion capture system is the more mainstream motion capture system.

The three-dimensional digital acquisition of Peking Opera movements is mainly to obtain the movement information of the main joints of the Peking Opera actors’ bodies through the motion capture technology, and then analyze and process the data to obtain more accurate motion capture sequence data, forming a Peking Opera standard movement database. Since the character costumes of Peking Opera are usually wide and fluttering-sleeved, it is difficult to capture by optical motion capture system, and at the same time, optical motion capture is prone to problems such as blocking of the body joints, which results in inaccurate motion capture data, while inertial motion capture system (referred to as inertial motion capture system) usually binds the sensors to the main joints of the actor’s body, which does not have the problem of blocking and the data is more accurate, therefore, this experiment mainly uses inertial motion capture technology to obtain motion information about the main body joints of actors, then analyze and process them to get more accurate motion capture sequence data. Therefore, this experiment mainly uses the inertial motion capture system to acquire three-dimensional digital images of Peking Opera movements. The inertial motion capture system measures the acceleration, orientation, angle, and other data of the key joints of the performer through multiple inertial measurement units, and then synthesizes the human motion data. In this paper, a full-body motion capture system is used to capture the movements of Peking Opera. The inertial motion capture system adopts wireless design, a single sensor is tied to the corresponding position of the human body joints through elastic straps, the wireless high-speed data is transmitted to the receiver, and then the data acquired by multiple sensors are summarized and transmitted to the computer for processing, the data of a single sensor includes sensor number, three-axis acceleration information, three-axis angular velocity information, etc.. The whole body acquires the motion information of the human body through 17 wireless inertial sensors, so that the body movements can be accurately captured. To realize the three-dimensional data acquisition and analysis of the Peking Opera form, the specific research process is shown in Figure 1.

Figure 1.

Specific research process

This chapter mainly analyzes the basic theories involved in motion capture systems, including commonly used coordinate systems, conversion between coordinate systems, principles of pose detection, commonly used data fusion algorithms and classification and recognition algorithms. The description of human posture in human motion capture systems has practical significance only under certain reference benchmarks, and the spatial coordinate system is precisely the reference benchmark. Frequently used spatial coordinate systems include carrier coordinate systems, geographic coordinate systems, navigation coordinate systems, geocentric coordinate systems, and so on.

The three attitude angles of the inertial measurement unit are: roll angle, pitch angle and yaw angle. Among them, the roll angle is the angle generated during left and right movement, the pitch angle is the angle generated during forward and backward movement, and the yaw angle is the angle generated during up and down movement. Therefore, using the three angle information of roll angle, pitch angle, and yaw angle, the state of the inertial measurement unit, and thus the human body’s attitude, can be represented.

In the process of transforming the navigation coordinate system into the carrier coordinate system, the origin of the navigation coordinate system and the carrier coordinate system are made to coincide, and the navigation coordinate system is rotated three times around the three coordinate axes to obtain the carrier coordinate system, and the angle of rotation is called the attitude angle or Euler angle of the carrier. 1)

Yaw angle α is the angle of rotation about axis z, range [−π, π].

In the transformation process, the navigation coordinate system is first rotated about the z-axis, then about the x-axis, and finally about the y-axis to obtain the carrier coordinate system: (1){x′=xcosα−ysinαy′=xsinα+ycosαz′=z$$\left\{ {\begin{array}{l} {x' = x\cos \alpha - y\sin \alpha } \\ {y' = x\sin \alpha + y\cos \alpha } \\ {z' = z} \end{array}} \right.$$ (2){x′=xy′=ycosβ+zsinβz′=−ysinβ+zcosβ$$\left\{ {\begin{array}{l} {x' = x} \\ {y' = y\cos \beta + z\sin \beta } \\ {z' = - y\sin \beta + z\cos \beta } \end{array}} \right.$$ (3){x′=xcosγ−zsinγy′=yz′=xsinγ+zcosγ$$\left\{ \begin{array}{l} x' = x\cos \gamma - z\sin \gamma \\ y' = y \\ z' = x\sin \gamma + z\cos \gamma \\ \end{array} \right.$$

The rotation formula around the z-axis is shown in Eq. (1), the rotation formula around the x-axis is shown in Eq. (2), and the rotation formula around the y-axis is shown in Eq. (3). From this, the rotation matrices of the three rotations can be obtained as: (4)Cn1=[cosα−sinα0sinαcosα0001]$$C_n^1 = \left[ {\begin{array}{*{20}{c}} {\cos \alpha }&{ - \sin \alpha }&0 \\ {\sin \alpha }&{\cos \alpha }&0 \\ 0&0&1 \end{array}} \right]$$ (5)C12=[1000cosβsinβ0−sinβcosβ]$$C_1^2 = \left[ {\begin{array}{*{20}{c}} 1&0&0 \\ 0&{\cos \beta }&{\sin \beta } \\ 0&{ - \sin \beta }&{\cos \beta } \end{array}} \right]$$ (6)C2b=[cosγ0−sinγ010sinγ0cosγ]$$C_2^b = \left[ {\begin{array}{*{20}{c}} {\cos \gamma }&0&{ - \sin \gamma } \\ 0&1&0 \\ {\sin \gamma }&0&{\cos \gamma } \end{array}} \right]$$

Therefore, based on the rotation matrix of three times, it is possible to obtain the rotation matrix from the navigation coordinate system to the carrier coordinate system as: (7)Cnb=C2bC12Cn1=[cosγcosα+sinγsinβsinα−cosγsinα+sinγsinβcosα−sinγcosβcosβsinαcosβcosαsinβsinγcosα−cosγsinβsinα−sinγsinα−cosγsinβcosαcosγcosβ]$$\begin{array}{l} C_n^b = C_2^bC_1^2C_n^1 = \\ \left[ {\begin{array}{*{20}{c}} {\cos \gamma \cos \alpha + \sin \gamma \sin \beta \sin \alpha }&{ - \cos \gamma \sin \alpha + \sin \gamma \sin \beta \cos \alpha }&{ - \sin \gamma \cos \beta } \\ {\cos \beta \sin \alpha }&{\cos \beta \cos \alpha }&{\sin \beta } \\ {\sin \gamma \cos \alpha - \cos \gamma \sin \beta \sin \alpha }&{ - \sin \gamma \sin \alpha - \cos \gamma \sin \beta \cos \alpha }&{\cos \gamma \cos \beta } \end{array}} \right] \\ \end{array}$$

Since the rotation matrices of the three times are orthogonal, Cnb$$C_n^b$$ is also orthogonal, so the rotation matrix from the carrier coordinate system to the navigation coordinate system is: (8)Cbn=(Cnb)−1=(Cnb)T=[cosγcosα+sinγsinβsinαcosβsinαsinγcosα−cosγsinβsinα−cosγsinα+sinγsinβcosαcosβcosα−sinγsinα−cosγsinβcosα−sinγcosβsinβcosγcosβ]$$\begin{array}{l} {C_b^n = {{\left( {C_n^b} \right)}^{ - 1}} = {{\left( {C_n^b} \right)}^T} = } \\ {\left[ {\begin{array}{*{20}{c}} {\cos \gamma \cos \alpha + \sin \gamma \sin \beta \sin \alpha }&{\cos \beta \sin \alpha }&{\sin \gamma \cos \alpha - \cos \gamma \sin \beta \sin \alpha } \\ { - \cos \gamma \sin \alpha + \sin \gamma \sin \beta \cos \alpha }&{\cos \beta \cos \alpha }&{ - \sin \gamma \sin \alpha - \cos \gamma \sin \beta \cos \alpha } \\ { - \sin \gamma \cos \beta }&{\sin \beta }&{\cos \gamma \cos \beta } \end{array}} \right]} \end{array}$$

The measurement principles of accelerometers, gyroscopes and magnetometers are different, and the data obtained by each sensor is also different, but the carrier’s attitude at a certain moment is uniquely determined, so the whole system should also output uniquely and accurately attitude information in the end, so it is necessary to fusion process the data output from accelerometers, gyroscopes and magnetometers.

Long and short-term memory network (LSTM) adds store and forget function on the basis of traditional recurrent neural network, which can better solve the time series problem and can memorize long time historical data to improve the accuracy of prediction. Long and short-term memory network consists of four parts: input gate, forgetting gate, output gate, and cell state. The working process of long and short-term memory network is divided into three stages: 1)

In the forgetting phase, the forgetting gate generates a function f_t based on the output value h_t−1 of the previous memory block and the input value x_t at the current moment, and then decides what information to discard in conjunction with the cellular state C_t−1, which passes when f_t = 1 and is discarded when f_t = 0; the expression for function f_t is: (16)ft=σ(wxfxt+whfht−1+bf)$${f_t} = \sigma \left( {{w_{xf}}{x_t} + {w_{hf}}{h_{t - 1}} + {b_f}} \right)$$

Where σ - sigmoid function. 2)

In the update phase, the value updated by the input gate and the new candidate value formed by the memory gate are added to the cell state to complete the update. The update formula for the cell state is: (17)it=σ(wxfxt+whfht−1+bi)$${i_t} = \sigma \left( {{w_{xf}}{x_t} + {w_{hf}}{h_{t - 1}} + {b_i}} \right)$$ (18)Ct=ftCt−1+ittanh(wxcxt+whcht−1+bc)$${C_t} = {f_t}{C_{t - 1}} + {i_t}\tanh \left( {{w_{xc}}{x_t} + {w_{hc}}{h_{t - 1}} + {b_c}} \right)$$

f - Oblivion gate.

i - Input gate.

C - Cell state.

o - Output gate.

h - Hidden layer output.

w_x - Input weights.

w_h - Hidden layer weights.

b - Offset.

The human body is an extremely complex organism, the human body is composed of 206 bones, the parts connected between two bones become joints, to simulate and reproduce the human body’s movement posture, it is necessary to provide the posture data of the joints of each limb, due to the differences in the joints in the specific application of the joints need to be constrained and limited to certain joints, so that it is in a reasonable range of movement.

The characteristic correspondence of each joint is shown in Table 1. When describing the movement of the Peking Opera character model, it is required that the various parts of the Peking Opera character cannot undergo deformation in the process of completing the whole movement, so the body skeleton is idealized as a rigid body, and the internal structure of the rigid body does not change when it is subjected to external force during the movement of the rigid body. The motion process of the skeleton is simply understood as the translation plus rotation of the rigid body, and the joints are idealized as a sphere, and the movement of the Peking Opera character model is described as the relative position and attitude changes between the joints. Peking Opera character model as a graphical description of the form of the Peking Opera character, the movement posture data need to drive the Peking Opera character model movement to reproduce the Peking Opera character movement process, taking into account the large amount of data in the process of data transmission, the skeletal model of the Peking Opera character is abstracted and simplified into 17 joints According to the knowledge of the anatomy of the Peking Opera character, each piece of bone connected by the joints has its own position and movement attributes, and is subject to the joints’ specific constraints. The joints of Peking Opera characters are usually modeled as ball hinges, but the range of motion of each joint of a Peking Opera character is not as free as that of a hinge due to the physiological limitations of the movement, so it is necessary to clarify the constraints and limitations of each joint. The joints of Peking Opera characters are categorized into three types: rotary joints, hinge joints, and gimbal joints. Slewing joints have freedom in only one direction, and the direction of rotation is limited to a certain extent. Hinge joints possess degrees of freedom in both directions and can be used for twisting and flexing movements around the joint. The gimbal joint has a ball-and-socket-like structure and is able to rotate around it, thus having three degrees of freedom.

In this section, the database of Carnegie Mellon University is selected as the source of motion capture data, and BVH files are used as the carrier of motion capture data, and four basic movements, such as walking, running, jumping and cartwheel, are selected as the research objects respectively. Each type of action is captured by several different data captives, the motion capture data of one captive is selected for each type of action to construct the action pattern samples, the motion capture data of other captives are selected as the action test samples, and the key frames of the motion sequences are extracted by using the method given in this paper, and the actions of the test samples are recognized. The recognition accuracy of the action is introduced as an evaluation standard for action recognition.

Four basic actions such as walking, running, jumping and cartwheel are recognized respectively, and two methods, SegSVD and CDP, are chosen to compare with the method of this paper, and the recognition accuracy is used as a criterion to measure the performance of the algorithms. The recognition accuracy of the three algorithms for the four basic movements is shown in Table 2. From the table, it can be seen that the method of this paper can effectively recognize the four basic actions, and compared with the other two methods, the motion capture method of this paper has a high recognition accuracy, and the mean value of accuracy is 93%.

Using inertial motion capture technology equipment to collect 22 segments of Peking Opera Wushang movement data from 8 famous Peking Opera artists as well as 10 Tianjin Peking Opera Theater and China Academy of Theatre Arts Wushang students, a Peking Opera Wushang classical repertoire movement library was constructed. In this section, a variety of Peking opera performance movements or performance postures are analyzed, namely: horse stance, shaking the leaning flag, body posture, harrier turn (i.e., flip), rubbing steps, and crossing the legs and flinging the whiskers. The first three movements (horse stance, flag shaking, and posture) are very important basic movements in Peking Opera, while the last three movements (harrier turn, rubbing steps, and throwing the beard across the legs) are difficult “core kung fu” in Peking Opera performances. Through the analysis, it was found that each movement has certain valuable rules. Evaluation and analysis were carried out. Selected parts are introduced below. 1)

Analysis of the “Horse Stance” Movement

Horse stance is one of the most basic stances in basic kung fu. When squatting horse stance, the feet are separated by a distance slightly wider than shoulder width, and the whole person is in a semi-squatting posture. When squatting in the horse stance, one needs to use both legs muscles to maintain stability during the movement. Horse stance is a simple movement that can be measured by parameters such as knee angle and tailbone height. With the help of curve analysis tools, we quantitatively observed the pattern of change in tailbone height, knee angle, and other indicators, and found some details that are difficult to notice in traditional teaching.

The angle curve of the right knee of the “horse step” movement is shown in Figure 2, and we take the Peking Opera performance program “Qiba” as the analysis goal. In the captured “Qiba” data, there are two movements that show a clear horse step or half horse step posture, as shown in the blue and purple vertical lines in the figure. The figure shows the curve comparison of the height of the tail vertebrae between the Peking Opera teacher (red line) and the student (black line) when they complete the “Qiba” movement. The height of the teacher is 5 cm higher than that of the student, and the height of the teacher’s tail vertebrae is greater than that of the student in the upright position. However, the curve shows that the height of the student’s tail vertebrae is often higher than that of the teacher during the horse step. When completing the “Qiba” movement, the student’s height change rate was 0.56, which was higher than the teacher’s value of 0.22. This suggests that the students’ movements are less stable than the teacher’s, and this finding is also consistent with the conclusion obtained by observing the curve.

Figure 2.

The average height curve of the spine from the ground

Figure 3.

Spinal rotation acceleration curves in the “Qiba”

Figure 4.

The vertical angle curve of the spine in “The Harrier Turns Over”

Reasonable, scientific, systematic, and matching the form optimization training of the performance course is an effective way to improve students’ performance skills. 1)

Carry out diversified form and body basic training. Motion capture technology helps students to change the randomness of limbs, standardize students’ bad body posture, relax students’ stiff bodies, and prompt students to make clear use of the body as a creative tool for performance, control themselves, and find the most appropriate body posture to express their characters.