A study of the influence of audio signal processing technology on the expression of music aesthetics in piano performance

Frequency is the number of times a signal changes in one second of time, pitch is a person’s perception of the frequency of the sound, which is generally known as pitch in music, and tone is represented by the frequency shift, which is a direct correlation. Frequency, pitch control relationship is shown in Table 1.

Sound intensity refers to the strength of the main tone in the sound signal, is the basis for discriminating music, the strength of the sound is an objective physical quantity. The subjective amount of sound intensity reflected by the human ear is called “loudness”, which is the property of hearing to judge the strength of the sound.

People’s subjective feeling of the spectral characteristics of the composite sound is collectively called timbre. In the sound, timbre is mainly related to the harmonic components of the sound, the nature of the sound is a vibration, in the composition of the composite sound of a variety of frequency components, the lowest intrinsic frequency is called the fundamental frequency, the composition of the composite sound of the rest of the frequency is equal to the fundamental frequency is called a harmonic multiple of the frequency.

The more harmonic components contained in the audio signal, the better the tone. In piano playing, the quality of the sound is measured in terms of the frequency paradigm of the harmonic components contained in the sound signal, i.e. the bandwidth, which describes the range of frequencies that make up the composite signal.

Audio from the sound to be detected by computer equipment in the process of attenuation, attenuation greater than 6dB, so after the completion of the test audio must be tested to compensate for this compensation process becomes the signal aggravation. In digital signal processing is generally used based on the differential equation of the digital filter to achieve the signal aggravation: (1)y(n)=s(n)−αs(n−1)α∈[0.9,1.0]$$y(n) = s(n) - \alpha s(n - 1)\quad \alpha \in [0.9,1.0]$$

Most of the frame delimitation of the audio signal uses the Hemming window, after the pre-emphasis of the audio signal to take the frame processing, the frame length of L can generally take the value of 256 points or more, the frame shift of D can generally be taken as 64 or with the length of the frame. If an audio signal sequence is S(n), then the sequence after the sub-frame processing, the nth point in the lth frame and the relationship between the original audio sequence S(n) is: (2)xl(n)=s[(l−1)*D+n]n∈[0,255]$${x_l}(n) = s[(l - 1)^*D + n]\quad n \in [0,255]$$

where x_l(n) is the value corresponding to point n on frame l.

The purpose of endpoint detection is to determine the boundaries of the audio unit to be recognized, which in turn reduces the amount of recognition operations, eliminates redundant information and improves the recognition efficiency.In this paper, we use the short-time average amplitude to estimate the audio energy.

The idea of the average amplitude method is to take the absolute value of all the points in the region identified by the moving window and then sum it up to get an average. And the method to determine the threshold of the method, take the sound before the known as static 10 consecutive frames of data as the basis for calculating the minimum energy threshold I_thl and the maximum energy threshold I_thh determined by taking the maximum average amplitude of the 10 frames A_max and the minimum average amplitude A_min and then determine the high and low thresholds through the formula (3): (3){Ia=0.03*(Amax−Amin)+AminIb=4*AminIthl=MIN(Ia,Ib)Ithh=5*Ithl$$\left\{ {\begin{array}{*{20}{c}} {{I_a} = {{0.03}^*}({A_{\max }} - {A_{\min }}) + {A_{\min }}} \\ {{I_b} = 4^*{A_{\min }}} \\ {{I_{thl}} = MIN({I_a},{I_b})} \\ {{I_{thh}} = 5^*{I_{thl}}} \end{array}} \right.$$

The method of determining the starting point according to the threshold is: according to I_thl and I_thh, take the first point S(i) that exceeds the value of I_thl, and locate the frame number where S(i) is located to reach the frame number that reaches the average amplitude first, but after a specified period of time, the frame amplitude does not reach I_thh and then decreases to I_thl again, then the frame where this S(i) is located can not be used as a boundary frame, and the system will return to the original state to make a new selection, until there is a frame of S(i) that exceeds the high threshold for a period of time after exceeding the low threshold, then this frame is the starting frame of audio signal recognition.

The core idea of linear prediction is that, based on the linear correlation between a point in the audio signal and a number of points preceding it, a linear combination of a series of points preceding that point is then applied to represent the information at that point, called a prediction. The error is then processed by taking the difference between the predicted point and the actual point as the error, which in turn yields a series of correlation coefficients. The prediction error e(n) according to the principle of linear correlation is expressed as: (4)e(n)=s(n)−∑i=0Nais(n−i)$$e(n) = s(n) - \sum\limits_{i = 0}^N {{a_i}} s(n - i)$$

In Eq. (4), N is the linear prediction order, which is generally chosen to be 18th order. In order to obtain the optimal solution for a_i, e(n) must be minimized. The method generally used is the minimum mean square error criterion to determine. The short time average error in a frame is: (5)E{e2(n)}=E{[s(n)−∑i=0Nais(n−i)]2}$$E\{ {e^2}(n)\} = E\{ {[s(n) - \sum\limits_{i = 0}^N {{a_i}} s(n - i)]^2}\}$$

Making the value of the partial derivative of this error with respect to a_i to be 0 can be obtained: (6)E{[s(n)−∑i=0Nais(n−i)]s(n−j)}=0$$E\{ [s(n) - \sum\limits_{i = 0}^N {{a_i}} s(n - i)]s(n - j)\} = 0$$

where i, j ∈ [1, N], N is the maximum prediction order.

From the above equation, it can be seen that when the linear prediction coefficients are taken to the optimal value, the prediction error e(n) is orthogonal to the past sample points, and out of the short-term smoothness of the audio signal is known that for N each audio segment S_n in a frame, there is Φ_n(i, j) for: (7)Φn(i,j)=E[Sn(m−i)Sn(m−j)]$${\Phi _n}(i,j) = E[{S_n}(m - i){S_n}(m - j)]$$

Then there is: (8)∑j=1pajΦn(i,j)=Φn(i,0)$$\sum\limits_{j = 1}^p {{a_j}} {\Phi _n}(i,j) = {\Phi _n}(i,0)$$

Let the autocorrelation function of the audio segment be: (9)RN(i)=∑n=iN−1SN(n)SN(n−j)$${R_N}(i) = \sum\limits_{n = i}^{N - 1} {{S_N}} (n){S_N}(n - j)$$

The autocorrelation method is applied to solve the equation for the optimal coefficients as: (10)[RN(0)RN(1)⋯RN(p−1)RN(1)RN(0)⋯RN(p−2)⋮⋮⋱⋮RN(p−1)RN(p−2)⋯RN(0)][a1a2⋮ap]=[RN(1)RN(2)⋮RN(p)]$$\left[ {\begin{array}{*{20}{c}} {{R_N}(0)}&{{R_N}(1)}& \cdots &{{R_N}(p - 1)} \\ {{R_N}(1)}&{{R_N}(0)}& \cdots &{{R_N}(p - 2)} \\ \vdots & \vdots & \ddots & \vdots \\ {{R_N}(p - 1)}&{{R_N}(p - 2)}& \cdots &{{R_N}(0)} \end{array}} \right]\left[ {\begin{array}{*{20}{c}} {{a_1}} \\ {{a_2}} \\ \vdots \\ {{a_p}} \end{array}} \right] = \left[ {\begin{array}{*{20}{c}} {{R_N}(1)} \\ {{R_N}(2)} \\ \vdots \\ {{R_N}(p)} \end{array}} \right]$$

The correlation coefficient can be solved by solving equation (10) and applying the Levinson-Durbin recursive algorithm.

The process of solving the cepstrum coefficient is to take the discrete Fourier transform of the audio signal and take the logarithmic sequence of the transformed sequence, and then take the discrete Fourier inverse transform of this logarithmic sequence. The cepstrum coefficients can be obtained by using Eq: (11)Cn={an+∑k=1N−1kCkaN−kn,i≤n≤pan+∑k=N−pN−1kCkaN−kn,n>p$${C_n} = \left\{ {\begin{array}{*{20}{c}} {{a_n} + \sum\limits_{k = 1}^{N - 1} {\frac{{k{C_k}{a_{N - k}}}}{n}} ,i \leq n \leq p} \\ {{a_n} + \sum\limits_{k = N - p}^{N - 1} {\frac{{k{C_k}{a_{N - k}}}}{n}} ,n > p} \end{array}} \right.$$

In Eq. (11), C_i is the cepstrum coefficient, a_i is the linear prediction coefficient, and p is the order of the cepstrum coefficient.

The MFCC is solved by first performing a short-time Fourier transform [21-22] on the audio signal to obtain the frequency spectrum, and then squaring the amplitude of the frequency spectrum to obtain the energy spectrum. Next, a set of triangular wave filters is applied to filter the energy spectrum in the frequency domain. The center frequency of each triangular wave in the triangular filter set is uniformly arranged according to the frequency scale, and the relationship between the Mel frequency scale and the Fourier frequency scale is: (12)M=2596*ln(1+f/700)$$M = 2596^*\ln (1 + f/700)$$

Where M is the Mel frequency in Mel and f is the frequency characterized by Fourier transform in Hz.

The number of filters is usually selected according to the number of critical bands, set the number of triangular filters for M, the filtered output is expressed as X(k), of which k ∈ [1, m]. The output of the filter bank to go to the logarithm, and then do the Fourier inverse transform can be obtained. This transformation process can be expressed as: (13)Cn=2M∑k=1MlogX(k)cos[(k−0.5)πn/M]$${C_n} = \sqrt {\frac{2}{M}} \sum\limits_{k = 1}^M {\log } X(k)\cos [(k - 0.5)\pi n/M]$$

Of these, n = 1, 2, …, L. Generally speaking, the number of MFCC coefficients for audio signals is selected from 12 to 16.

The basic idea of wavelet transform [23-24] is to represent or approximate a signal or function by a set of orthogonal functions, and this family of functions is called the wavelet function family, which is constituted by the translation and expansion of a basis wavelet function.

Let ψ(t) ∈ L²(R), L²(R) denote the square-productible real number space, i.e., the space of signals with finite energy, and its Fourier transform ψ^(ω)$$\hat \psi (\omega )$$, when ψ^(ω)$$\hat \psi (\omega )$$ satisfies the permissibility condition: (14)Cψ=∫k|ψ^(ω|2|ω|dω<∞$${C_\psi } = \int_k {\frac{{{{\left| {\hat \psi (\omega } \right|}^2}}}{{\left| \omega \right|}}} d\omega < \infty$$

Call ψ(t) a basis wavelet (or wavelet basis function). Taking the basis wavelet ψ(t) and stretching (scaling) and translating (time shifting) it gives a wavelet sequence: (15)ψa,b(t)=|a|−1/2ψ((t−b)/a)a,b∈R,a≠0$${\psi _{a,b}}(t) = {\left| a \right|^{ - 1/2}}\psi \left( {(t - b)/a} \right)\quad a,b \in R\:,a \ne 0$$

where a is called the scale factor and b is called the translation factor.

Let ψ(t) be the basis wavelet and ψ_ab(t) be the continuous wavelet sequence obtained after scaling and translation of the wavelet basis function, then the continuous wavelet transform (CWT) of signal f(t) ∈ L²(R) is defined as: (16)(Wψf)(a,b)=〈f,ψa,b〉=|a|−1/2∫ℝf(t)ψ((t−b)/a)¯dt$$({W_\psi }f)(a,b) = \left\langle {f,{\psi _{a,b}}} \right\rangle = {\left| a \right|^{ - 1/2}}\int_\mathbb{R} f (t)\overline {\psi ((t - b)/a)} dt$$

where ψa,b(t)¯$$\overline {{\psi _{a,b}}(t)}$$ is the complex conjugate of ψ_a,b(t).

Any transformation should have an inverse transformation for it to have any application. Let ψ(t) be a fundamental wavelet and for any f(t), g(t) ∈ L²(R), there is: (17)∫−∞∞∫−∞∞(Wψf)(a,b)(Wψf)(a,b)¯1a2dadb=Cψ〈f,g〉$$\int_{ - \infty }^\infty {\int_{ - \infty }^\infty {({W_\psi }f)(a,b)} } \overline {({W_\psi }f)(a,b)} \frac{1}{{{a^2}}}dadb = {C_\psi }\langle f,g\rangle$$

Moreover, when the Fourier transform ψ^(ω)$$\hat \psi (\omega )$$ of the fundamental wavelet ψ(t) satisfies the permissibility condition, the original signal f(t) can be recovered from the continuous wavelet transform if the signal f(t) is continuous at t ∈ R, i.e., the inverse transformation of the continuous wavelet transform exists: (18)f(t)=1Cψ∫∞∞∫∞∞(Wψf)(a,b)ψa,b(t)1a2dadb$$f(t) = \frac{1}{{{C_\psi }}}\int_\infty ^\infty {\int_\infty ^\infty {({W_\psi }f)(a,b)} } {\psi _{a,b}}(t)\frac{1}{{{a^2}}}dadb$$

In ψ_a.b(t) the variable a reflects the scale of the function and the variable b detects the position of the translation along the time axis or position draw. In general, the energy of the basis function ψ(t) is concentrated at the origin and the energy of the continuous wavelet function ψ_a,b(t) is concentrated at point b.

In practical applications of wavelet transforms, especially in wavelet analysis of signals on computers, continuous wavelet transforms must be discretized in continuous wavelet transforms, continuous wavelet functions are considered: (19)ψa,b(t)=|a|−1/2ψ(t−ba)$${\psi _{a,b}}(t) = |a{|^{ - 1/2}}\psi (\frac{{t - b}}{a})$$

For simplicity, it is assumed here that the conditions: a ∈ R⁺, b ∈ R and a ≠ 1, the Fourier transform ψ^(ω)$$\hat \psi (\omega )$$ of the wavelet basis function ψ(t) satisfy the permissibility condition. Then the permissibility condition is changed to under the assumption:

Reconstruction formula for the discretized wavelet transform: (20)f(t)=c∑−∞∞∑−∞∞Cj,kψj,k(t)$$f(t) = c\sum\limits_{ - \infty }^\infty {\sum\limits_{ - \infty }^\infty {{C_{j,k}}} } {\psi _{j,k}}(t)$$

Here, c is a constant independent of the original signal.A discretized wavelet ψ_j,k(t) = 2 ^{− j/2}ψ(2 ^{− j}t − k)(j, k ∈ Z) with the discretization parameter taken as a₀ = 2, b₀ = 1 is often called a binary wavelet.

The tree structure of the three-level multiresolution analysis of Signal S is shown in Figure 2:

Figure 2.

Tree structure of multi-resolution analysis

Multi-resolution analysis knowledge further decomposes the low frequency portion of the signal, while the high frequency portion is not decomposed, and the decomposition has the relation: S = A3 + D3 + D2 + D1.

By multiresolution analysis and the theory of spatial orthogonal decomposition, L2(R)=∑j=−∞Wj⊕Vs$${L^2}(R) = \sum\limits_{j = - \infty } {{W_j}} \oplus {V_s}$$, (s is an arbitrarily set scale). If the signal f(t) ∈ L²(R) is decomposed in space L²(R), the following decomposition expression can be obtained: (21)f(t)=∑j=−∞s∑k=−∞+∞dj,kψj,k(t)+∑k=−∞+∞cs,kψs,k(t)$$f(t) = \sum\limits_{j = - \infty }^s {\sum\limits_{k = - \infty }^{ + \infty } {{d_{j,k}}} } {\psi _{j,k}}(t) + \sum\limits_{k = - \infty }^{ + \infty } {{c_{s,k}}} {\psi _{s,k}}(t)$$