The Effects of Intelligent Semantic Analysis Techniques on Language Acquisition in the Improvement of English Intercultural Communication Skills

Lexical analysis is the process of taking apart each word in a sentence individually and analyzing its lexical nature, lexical meaning, etc. The lexical analysis of utterance is mainly based on lexical analysis, which first obtains the lexical nature and lexical meaning of each word, and then, based on the obtained data and the lexical nature of the words before and after the word, it determines the lexical nature of the word that the word plays in this utterance.

The key problem that needs to be solved in cross-language dependent syntactic analysis is the variability of the two languages themselves, the source language and the target language, both at the word level and at the sentence structure level. This variability is mainly due to the fact that we use different words to express the same meaning in different languages, and each language has its own grammar to constrain the order of occurrence of words [19]. The representation learning method for language independent features is to formulate the same feature rules among different languages to represent words from multiple languages in the same feature space. Linguistically independent feature representation learning is mainly used to compensate for the differences between languages and establish inter-language connections by inducing the generation of linguistically independent features, and then train the dependent syntactic analyzer through the vector representation of the source language in the feature space, so that the dependent syntactic analyzer can carry out the dependent syntactic analysis of the target language.

Each word pair pair = 〈E_i,F_i〉 in the set of bilingual word pairs V represents the equivalence of English word E_i and Chinese word F_i. After constructing the set of bilingual word pairs V, we will learn to obtain a vector representation for each word pair Pair in the set V by utilizing the deep neural network approach. This vector representation is capable of representing any of the words in the word pair simultaneously. The model used for the learning process is shown in Figure 2.

Figure 2.

Representation learning of word embedding

The input layer of the model shown in Figure 2 is introduced first. From the corpus of the source language, a clause X m is selected as the representation of word w_i centered on word W_i by a specified window length C. A window length of C means to w_i must be centered on W₁W₂......W_c. Then for each word W_i in the words X obtain a representation of the word direction of the word, denoted as R(W_i) . Splice each R(W_i) together m to form a R(X) matrix as in equation (1): (1) R(X)=[ R(W1);R(W2);R(W3)……R(Wc) ]

After obtaining the R(X) matrix, each word in clause X has a corresponding vector representation in the R(X) matrix. The R(X) matrix can then be seen as a representation of clause X. We use R(X) as the input to the deep neural network model. Next, the neural network model will be based on the current input R(X) and the output obtained by nonlinearly transforming the input R(X) using the tanh function of the hidden layer of Fig. 2 is H₁(X), as shown in (2): (2) H1(X)=tanh(W1×R(X)+b1)

where W₁ is the weight of matrix R(X) and b₁ is the bias. H₁(X) is the output vector of the first hidden type. From the model in Fig. 2, it can be seen that the output of each hidden layer will be used as the input of the hidden type behind it, which is then transformed by a nonlinear transformation to get a new output again. The exact number of hidden layers is set dynamically in: the experiment, for example, the output of the i th hidden is H_i(X) and its input is H_i–1(X), as shown in (3): (3) Hi(X)=tanh(Wi×Hi−1(X)+bi)

Similarly, where w_i is the weight matrix of the current layer, b_i is the bias, and H_i(X) is the output of the i th hidden layer. Once the number of hidden layers t in the model is determined, the output of the last layer is needed to calculate a final score S(X).

The parameters included in equations from (1) to (4) are the hidden layer parameter { Wi,bi }i=1t and the final output layer parameter (θ,μ). The above is the representation process of word vectors, and the next is the parameter solving process. In the representation learning process we usually utilize the binary classification problem to solve the parameters. First, construct the positive negative class training set D={ xi,xi− }i=1N , where x_i = W₁W₂......W_c, i.e., the clauses extracted by a fixed window length C, are used as positive samples. And its negative sample xi− is obtained by replacing the center word of x_i with the word paired with x_i in the set of bilingual word pairs V. We input each of the positive and negative samples in D into the model in Fig. 2 and obtained two scores S(x_i) and S(xi−) . For solving the parameters we defined the loss function as shown in (5): (5) J(D)=1N∑i=1N(0,1−S(xi)+S(xi−))

The stochastic gradient descent method is utilized to solve the above equation, and the values of the model parameters are continuously updated over several iterations. By training the above model we obtain the word embedding matrix R for all words in both languages, and each word pair in the set of bilingual word pairs V shares the same language-independent word vector representation R(W_i). In order to make the dependency syntactic analyzer trained on the source language side work on the target language side, we train a non-lexicalized dependency syntactic analyzer by replacing the specific words in the English sentences on the source language side by the international common lexical tags corresponding to each word together with its word vector representations R(W_i) as the input.

The vector space modeling criterion is used in measuring utterance similarity. Vector space modeling involves separating the smallest semantic units such as words and phrases in a text and using their calculated similarity as vector elements. Teaching cosine is used in two English sentences to obtain the inter-semantic similarity.

In the vectorized representation of English utterances, two utterances are first represented as equal length vectors, e.g., for utterances T₁ and T₂, all the words of the two utterances are assembled into a joint word set T as in equation (6).

Removing the same words in T₁ and T₂ ensures the mutual dissimilarity of the elements in the union set T, where {w₁,w₂,…w_m} is the set of words in statement T₁ and {q₁,q₂,…,q_n} is the set of words in statement T₂, e.g., for English statements:

T₁:{What is your favorite sports?}

T₂:{What kind of sports do you enjoy most?}

Combine the two statements by removing the articles and exclamations from both statements, keeping the real word prototypes, and recording the same words to obtain the combined statement T: {What is your favorite sport kind of do you enjoy most?}. Represent the joint utterance T as a vector S. The length of the words of the joint semantic vector is the same as the number of joint utterances, and at the same time, the utterance T₁ is denoted by asking the joint semantic vector S₁, and T₂ is denoted by asking the joint semantic vector S₂. The words in the vector are in the form of component values, and S_i is taken to be 1 if S_i is included in the semantic vector, and the similarity is computed according to the equation (1) if it is not included in the utterance. Obtain semantic vectors S₁ and S₂ corresponding to utterances T₁ and T₂.

where the decimal obtained from the calculation is the value of similarity corresponding to the word. The prepositions and auxiliary verbs such as of, do, etc. in the utterances have no comparative words. After determining the semantic vectors S₁ and S₂ corresponding to statements T₁ and T₂, the similarity of T₁ and T₂ is calculated according to Eq. (7), and the alternative answer with a similarity value greater than the set threshold is taken as the final answer as in Eq. (8).

LSTM is widely used in various tasks in the field of natural language processing, and its structure based on recurrent neural networks allows the model’s output at a certain positional moment to carry information with simultaneous context [20]. The network structure of LSTM is similar to that of RNN, in that it is a repetitive module connected in a recurrent manner, and unlike the RNN, the LSTM has four internal neural network layers that interact in a very specific way, and its internal network structure is shown in Figure 3.

Figure 3.

LSTM inner structure

The LSTM has three structures called “gates”, which consist of a sigmoid neural network layer and an element-by-element multiplication operation, which represents a method of selectively letting information pass through, the element-by-element multiplication means multiplying the input by a number in the interval 1, where 0 means it cannot pass through, and (0,1) means it passes through all of them.

The key to the LSTM is the cell state C_t, which in the diagram is the horizontal line running across the top of the diagram, it represents the state of the input sequence up to the current point in time of the network, it goes through two element-by-element operations at each repetition of the module, the first element-by-element multiplication is done to select which positions in the previous state need to be forgotten and which need to be remembered; the second element-by-element addition will determine what new information will be added to the cell state [21]. The formula here is shown in (13): (13) Ct=ft⊗Ct−1⊕it⊗C˜t

In the above equation, C˜t represents the new information, i_t represents the filtering of the new information and they are calculated as shown in Equation (14) and Equation (15), x_t is the input at the current point in time, σ is the sigmoid function, which is calculated as shown in (16), and the output is the value in the interval of (0,1), which is used for filtering the information: (14) it=σ(Wi[ ht−1;xt ]+bi) (15) C˜t=tanh(Wc[ ht−1;xt ]+bc) (16) σ(x)=11+e−x

Finally the network will determine the value of the output based on the cell state, the output of the previous time point and the input of the current time point, calculated as shown in Eqs. (17) and (18), where o_t denotes the result of the output gate, and finally the final hidden output h_t is obtained by doing an element-by-element multiplication with the cell state C_t: (17) ot=σ(Wo[ ht−1;xt ]+bo) (18) ht=ot⊗tanh(Ct)

The stack-LSTM adds a stack pointer to the normal LSTM. similar to the traditional LSTM, new inputs are always added to the last position, but when computing the new memory part, the stack pointer determines from which position of the LSTM to provide the c_t–1 and h_t–1 needed for the computation. In addition to adding new elements to the end of the sequence, the stack-LSTM provides the action of bouncing the stack by pointing the stack pointer to the previous element. Thus, this stack-LSTM can be viewed as a stack, where the press-stack action places the new element at the end of the sequence, saves a pointer to the current top of the stack, and then points the top-of-stack pointer to the new element, and the pop-stack action simply updates the top-of-stack pointer without performing additional operations. The output of the layer pointed to by the top-of-stack pointer is a vector representation of all the contents of the stack in continuous space.