A Study of the Effectiveness of Using Deep Learning Algorithms to Analyze Legal Risk Identification in Social Work Programs

Social work project refers to the general term for activities in the field of social work services in which social work agencies and social workers apply professional theories and methods, reasonably utilize internal and external resources, and carry out social work services in a planned and purposeful manner according to the project management process, to satisfy the personal needs of the service recipients, to integrate community resources, and to solve social problems. Most of the current social work programs come from government-purchased services, and the execution and implementation of these programs depend on the cooperation between government agencies and social workers.

The organizational framework of social work agencies is shown in Figure 1. Social work agencies are committed to professional values, theories and skills of social work, serving the needy people in society, adhering to standardized management, project-based operation, participatory innovation, high-performance outputs, and aiming at personal empowerment, community empowerment, and professional enrichment. The scope of the agency’s services mainly includes children’s social work, youth social work, gerontological work, social work for the disabled, and the main source of projects is government-funded services.

Figure 1.

Organization framework of social work agencies

The way in which judicial administrative bodies obtain social work services is through the purchase of services and the commissioning of projects. Prior to the promulgation and implementation of the Law, most regions purchased services from “social workers”. Most of these “social workers” did not possess the professional qualifications of professional social workers, lacked experience in the practice of social work, and were mainly engaged in assisting the judicial administrative organs in carrying out administrative support services, most of the content of which had nothing to do with social work, and were in fact of the nature of temporarily employed persons. Although they are also referred to as “social workers”, this is merely a term used by the judicial administrative organs to distinguish them from other temporary staff, and they are not “social workers” in the professional sense. In addition, even professionals who are assigned by social work service organizations to work in judicial administrative organs are susceptible to interference by administrative forces in their work, and their professional independence is easily limited. Project entrustment, also a kind of purchased service, is a system whereby the judicial administrative organs entrust the educational and supportive work of community corrections in the form of “project packaging” to professional organizations and pay the fees, with the professional organizations participating in the whole process of project design, implementation and acceptance, and carrying out the services as a package. The content of the work entrusted to the project is usually highly specialized, requiring overall assessment, overall budgeting and overall design, and is able to make up for the lack of specialization on the part of the community corrections staff of the judicial administrative organs.

In the process of social work implementation, on the one hand, it is due to the hierarchical relationship between social work organizations and the government. On the other hand, it is the complexity and diversity of the social work program itself, which leads to the problems of the legal system being difficult to implement, the legal risks to be solved and the regulatory system to be improved in the social work program. Therefore, this paper will focus on the identification of legal risks in social work programs.

The legal risks of social work programs can be mainly classified into violation risks, exercise risks, contract risks, tort risks, liability risks and other risks. Violation risk mainly refers to the legal sanctions or losses that may be suffered because of the violation of laws, statutes, regulations or provisions of the industry, and this risk may involve administrative liability, civil liability or even criminal liability. Contract risk is the legal risk that arises due to issues with contract signing and fulfillment. Exercise risk, in the social work program involves the handover of the upper and lower levels and the management of financial options, so the exercise risk is mainly due to the risk of not having enough funds above the cooperation and not being able to exercise the right. Infringement risk refers to the risk of infringing others’ intellectual property rights and labor rights in social work projects. Liability risk mainly refers to the negligent behavior of individuals or groups due to their involvement in the social work program. In addition to these other legal risks, this paper summarizes them as other risks.

Machine learning is a branch of artificial intelligence and also the core of artificial intelligence. The theory of machine learning focuses on giving computers the ability to learn by designing and analyzing algorithms, and is a simulation of the main cognitive processes of human beings, and machine learning has gone through the culmination of two developments: shallow learning and deep learning.

Among them, deep learning is to build a network structure similar to the structure of the human brain by stacking multiple implicit layers in order to simulate the process of human brain thinking and learning. Each layer of the network takes the features extracted from the previous layer as learning objects to discover the feature representation of the input information, thus realizing a more accurate model structure and improving the network’s ability to classify and predict. The most common deep learning methods are stacked self-encoders, convolutional neural networks, and deep confidence networks. Learning training methods can be classified as supervised and unsupervised depending on the sample.

The unsupervised learning property of self-encoders makes them commonly used for training and testing of unlabeled samples. In the structure it is assumed that the output value of the network is equal to the input value, i.e., input layer x = output layer x′, and then the neural network is trained and the back propagation algorithm is utilized to reshape the weight parameters and bias parameters to obtain the feature parameters of the input dataset. The auto-encoder contains three layer structure of input layer, hidden layer and output layer.

The running process of the autoencoder algorithm can be divided into two phases according to its function: encoding phase and decoding phase. From the input layer to the hidden layer is the encoding phase, the purpose is to downsize the input sample.

Encoding phase process: (1)y=f(ω1x+b1)$$y = f({\omega _1}x + {b_1})$$

where f denotes the activation function in the encoding stage. ω₁ denotes the weights between the input layer to the implicit layer. b₁ denotes the bias matrix. x denotes the input sample and y denotes the effective coding of the output.

From the hidden layer to the output layer is the decoding phase, which aims to remap the samples that have been dimensionalized to the original feature space again.

Decoding phase process: (2)x′=g(ω2y+b2)$${x'} = g({\omega _2}y + {b_2})$$

where g denotes the activation function in the decoding stage, ω₂ denotes the weights between the implicit layer and the output layer, b₂ denotes the bias, and x′ denotes the output of the decoding stage.

Self-encoder network training usually needs to be done by stochastic gradient descent (SGD) algorithm and error back propagation (BP) algorithm. The whole training process is a process of minimizing the reconstruction error, which can be expressed as a minimization cost function: (3)J=∑L(x,x')=−∑k=1n[xklogxk'(1−xk)log(1−xk')]$$J = \sum L(x,{x'}) = - \sum\limits_{k = 1}^n \left[ x_k \log x_k'(1 - x_k)\log (1 - x_k') \right]$$

As the training of the network is optimized, the output data x′ decoded from the effective coding of the output of the implicit layer y is gradually approached to the input samples x. When the training is over, the self-encoder extracts the features to obtain a new feature representation y.

Convolutional neural net is likewise a feed-forward deep neural network that utilizes convolutional budgets instead of matrix multiplication operations in network training, which reduces the computational difficulty of the network and reduces the risk of network overfitting, while at the same time improving robustness and stability. It generally consists of a stack of input, convolutional, pooling, fully connected and output layers.

The fully connected layer integrates the features extracted through convolution and pooling, filters the features of lower importance, retains the features of higher importance, connects each unit of the upper and lower layers, and finally outputs the labels. In order to improve the performance of the DNN network and avoid gradient explosion, the fully connected layer often uses the Relu function as the activation function. The output of the fully connected layer will be used as the input to the loss function of the network model to calculate the loss error.

The training of neural network is a process of constantly optimizing the loss function, the purpose is to make the loss value gradually become smaller, and finally reach a stable state. The commonly used loss functions include mean square error function, cross-entropy loss function, and quadratic cost function.

Mean square error function: (4)MSE=1n∑i=1n(yi−yi')2$$MSE = \frac{1}{n}\sum\limits_{i = 1}^n ( {y_i} - y_i'{)^2}$$

Cross-entropy loss function: (5)CLF=−1n∑x[ylna+(1−y)ln(1−a)]$$CLF = - \frac{1}{n}\sum\nolimits_x {\left[ {y\ln a + (1 - y)\ln (1 - a)} \right]}$$

Quadratic cost function: (6)QCF=12n∑x‖y(x)−aT(x)‖2$$QCF = \frac{1}{{2n}}\sum\limits_x {{{\left\| {y(x) - {a^T}(x)} \right\|}^2}}$$

Where, y_i denotes the actual data, yi′$$y_i^\prime$$ denotes the simulated data, x denotes the input samples, y denotes the actual classification labels, a, a^T denotes the output value of the network, n denotes the number of samples, and T denotes the maximum number of layers of the network.

The loss function can express the degree of similarity between the output labels and the real labels, so the loss function can be directly used to measure the strength of the DNN network training results. Optimizing the loss function algorithm can improve the accuracy of network recognition. Currently commonly used optimization algorithms are divided into: stochastic gradient descent, stochastic gradient descent with momentum, root mean square propagation and adaptive moment estimation. 1)

Stochastic gradient descent method

The standard gradient descent method is to update the network weights, bias and other parameters along the negative gradient direction of the loss function at each iteration to minimize the loss function. It can be expressed as: (7)φn+1=φn−α∇J(φn)$${\varphi _{n + 1}} = {\varphi _n} - \alpha \nabla J({\varphi _n})$$

where φ is the parameter vector, n is the number of network iterations, α denotes the learning rate, and J(φ_n) denotes the loss function.

Standard gradient descent algorithms need to utilize the entire dataset for computation, whereas stochastic gradient descent, the SGD algorithm mentioned above, needs to employ only a partial subset of the training data. 2)

Stochastic gradient descent with vectorization

Adding a momentum term to the gradient descent process can effectively reduce the problem of fluctuations. Stochastic Gradient Descent with Momentum (SGDM) can be expressed as: (8)φn+1=φn−α∇J(φn)+β(φn+1−φn)$${\varphi _{n + 1}} = {\varphi _n} - \alpha \nabla J({\varphi _n}) + \beta ({\varphi _{n + 1}} - {\varphi _n})$$

Where: β represents the effect of the previous descending gradient on the current iteration. 3)

Root mean square propagation method

The Root Mean Square Propagation method is a loss function optimization algorithm that uses the moving average of the squared gradient to dynamically adjust the learning rate. Different from the stochastic gradient descent method using a single learning rate, the root-mean-square propagation method can use different learning rates to continuously optimize the network training, which can automatically adapt to the loss function in the optimization process, accelerate the convergence speed of the model, improve robustness and avoid the problem of gradient disappearance. Its expression is: (9)θn=γ2θn−1+(1−γ2)[∇J(φn)]2$${\theta _n} = {\gamma _2}{\theta _{n - 1}} + (1 - {\gamma _2}){[\nabla J({\varphi _n})]^2}$$

where γ₂ is the decay rate of the moving average, which can be expressed as: (10)φn+1=φn−α∇J(φn)θn+ε$${\varphi _{n + 1}} = {\varphi _n} - \frac{{\alpha \nabla J({\varphi _n})}}{{\sqrt {{\theta _n}} + \varepsilon }}$$

where ε is an increasing constant 4)

Adaptive Moment Estimation

Adaptive Moment Estimation (Adam) is similar to the Root Mean Square Propagation (RMSP) method, the difference is that the RMSP algorithm uses a weighted average of the moving exponential of the squared gradient to adjust the learning rate, whereas Adaptive Moment Estimation (AME) takes into account both the mean and the variance of the gradient, and incorporates a momentum term to adaptively adjust the learning rate. Its expression is: (11)φn+1=φn−αmlθl+ε$${\varphi _{n + 1}} = {\varphi _n} - \frac{{\alpha ml}}{{\sqrt {\theta l} + \varepsilon }}$$

Among them: (12)mn=β1mn−1+(1−β1)∇J(φn)$${m_n} = {\beta _1}{m_{n - 1}} + (1 - {\beta _1})\nabla J({\varphi _n})$$ (13)θn=β2θn−1+(1−β2)[∇J(φn)]2$${\theta _n} = {\beta _2}{\theta _{n - 1}} + (1 - {\beta _2}){[\nabla J({\varphi _n})]^2}$$

where β₁, β₂ is the decay rate of the moving average.

Neural networks, also known as multilayer perceptrons, are one of the most basic neural network structures and the most widely used. A deep neural network (DNN) is a multilayer perceptron containing many (more than two) hidden layers.The DNN deep neural network is shown in Fig. 2 and consists of one input layer, three hidden layers, and one output layer. Here the input layer of the DNN is set as layer 0 and the output layer is set as layer L. The operations of the neural network can be defined as: (14)yl=f(xl)=f(Wlyl−1+bl),0<l<L$${y^l} = f({x^l}) = f({W^l}{y^{l - 1}} + {b^l}),0 < l < L$$

Figure 2.

Deep seeding network

Here x^l, y^l, W^l, b^l are the excitation vector, activation vector, weight matrix and bias vector of the tnd hidden layer respectively. y⁰ = o is the input feature vector of the network. f is an activation function that computes the excitation vector at the element level. The commonly used activation functions are:

The sigmoid function: (15)σ(x)=11+e−x$$\sigma (x) = \frac{1}{{1 + {e^{ - x}}}}$$

hyperbolic tangent function: (16)tanh(x)=ex−e−xex+e−x$$\tanh (x) = \frac{{{e^x} - {e^{ - x}}}}{{{e^x} + {e^{ - x}}}}$$

The hyperbolic tangent function is a tuned version of the sigmoid function and they have the same modeling capabilities. The difference is that the value domain of the sigmoid function is (0, 1), which helps to get a sparser representation. Whereas the value domain of tanh is (-1, 1) is symmetric and easier to train.

Rectified Linear Unit (ReLU): (17)ReLU(x)=max(0,x)$$\operatorname{Re} LU(x) = \max (0,x)$$

Its derivatives are more concise and do not suffer from gradient vanishing as the number of layers increases.

The selection of the output layer of a deep neural network usually varies depending on the task, and in the case of a regression task, a linear layer is usually used: (18)yL=xL=WLyL−1+bL$${y^L} = {x^L} = {W^L}{y^{L - 1}} + {b^L}$$

In case of multi-categorization tasks, it is common to use each neuron of the output layer to represent a category, where the value of the ist neuron represents the probability that the current feature belongs to category i, and thus it must satisfy a polynomial distribution, i.e., yiL≥0$$y_i^L \geq 0$$ and ∑i=1CyiL=1$$\sum\limits_{i = 1}^C {y_i^L} = 1$$, where C is the total number of categories. To satisfy this restriction, normalization is usually performed using the softmax function: (19)yiL=Pdnn(i|o)=softmaxi(xL)=exiL∑j=1CexjL$$y_i^L = {P_{dnn}}(i|o) = soft{\max _i}({x^L}) = \quad \frac{{{e^{x_i^L}}}}{{\sum\limits_{j = 1}^C {{e^{x_j^L}}} }}$$

Given the input of the feature vectors, the output of the DNN is determined by the model parameters M={Wl,bl,0<l≤L}$$\mathcal{M}=\{W^l,b^l,0< l \le L\}$$. The activation vectors as well as the final output are computed by using the previously mentioned formulas, a process also known as forward computation.

The DNN constructed in this paper consists of an input layer, a hidden layer, an output layer and a softmax function, where the input layer consists of four neurons corresponding to the four features in the IRIS dataset, which are used as input vectors, and the hidden layer has two layers, each consisting of five and six neurons, respectively, followed by an output layer, which consists of three neurons corresponding to the number of categories of the target variable in the IRIS dataset. Finally, there is the softmax function created for solving the multicategorization problem.

In this section, Pytorch deep learning framework is used as a tool to build DNN-based legal risk recognition model. The 1602 pieces of data are made according to the ratio of training set and test set as 2:1, and there are 1073 pieces of training set and 529 pieces of test set after making. Due to the small amount of data, the DNN network built in this paper is relatively simple, including three layers: the input layer, a hidden layer, and the output layer, in which the hidden layer contains 16 neurons, and the risk level used for legal risk assessment has three levels: high, medium, and low, so the category of the output layer is 3. The Softmax classifier is used to calculate the output probability of each category, and the ReLU activation function is used to prevent the gradient dispersion, to speed up the computation and increase the sparsity of the network, the function expression is: (20)F(x)=11+ex$$F(x) = \frac{1}{{1 + {e^x}}}$$ (21)f(u)=max(0,u)$$f(u) = \max (0,u)$$

Then the hyperparameters are set, the learning rate is set to 0.001, the number of iterations is set to 200, the number of batches processed each time is set to 50, the cross entropy loss function is selected for the loss function, and the training mode is retraining. After the setup is completed, the GPU is called to train the network, during the training process, we need to pay attention to the loss value (loss) and accuracy (accuracy) of the two parameters change, the loss value decreases to prove that the error is converging, the accuracy value increases to prove that the accuracy rate is increasing, these trends can indicate whether the generated model is ideal or not.

The change of loss curve of DNN risk recognition model built in this paper is shown in Fig. 3, from the change of loss value, we can see that the Train loss curve and Test loss curve, the change trend of the two curves is basically the same, starting from the loss value of about 1 to start a sharp decline, after 40epochs of iteration the curve begins to slow down the decline, and then to the After 100epochs the train loss and test loss curves gradually stabilize, indicating that the network begins to converge, and the training is completed at 200 Epochs, and the loss value drops to 0.1.

Figure 3.

Change in the loss curve

The change of the accuracy curve of the DNN risk recognition model is shown in Fig. 4, as can be seen from the accuracy curve, the training accuracy curve of the DNN network increases dramatically after the increase of the number of iterations accuracy, starts to become slower at about 20epochs and reaches about 90%, and with the hierarchical increment reaches the convergence at about 80epochs and the accuracy reaches about 98%. Thereafter it remains stable until the end of training. The test accuracy trend is roughly the same. From the loss curve and accuracy curve, it can be seen that the network convergence of the DNN legal risk identification model is stable, and from the accuracy rate and loss value, it can be seen that the generated model has better performance.

Figure 4.

Change in the accuracy curve

Common model evaluation metrics include precision rate, recall rate, and F1 value. Precision rate is the number of key elements correctly identified by the model (TP), as a proportion of all identified key elements (TP+FP). Where FP is the number of misidentified key elements. Recall is the ratio of the number of key to be (TP) correctly identified by the model to all true key elements (TP+FN). Where FN is the number of non-critical elements that were incorrectly identified.

In the previous section the risk identification model is constructed and trained to verify its performance. The next section will validate the effectiveness of the legal risk identification model of this paper in identifying legal risks in social work programs. A total control group was designed to verify its feasibility and effectiveness in the constructed legal risk dataset of social work programs, and HMM, CRF, BiLSTM, LSTM, BiLSTM-CRF, and BERT-CRF were used for comparison. The results of the experimental comparison are shown in Fig. 5, from which it can be seen that this paper’s deep learning-based legal risk recognition model achieves optimal results in all three metrics, with precision, recall, and F1 values of 89.32%, 86.23%, and 87.35%, respectively. Outside of the first time, the BiLSTM-CRF model and BERT-CRF model have relatively high recognition rates, with recognition precision rates of 87.25% and 88.12%, respectively.

Figure 5.

Experimental comparison

The legal risk elements of social work programs are classified into six types, including violation risk (S1), exercise risk (S2), contract risk (S3), tort risk (S4), liability risk (S5), and other risks (S6). In the identification of legal risks in social work programs, the identification of legal risk elements by different models is shown in Table 1. From the situation of the identification accuracy of the six legal risk elements, the average accuracy of the risk identification of the model in this paper is 88.152%, which is higher than the other models. In terms of risk identification, the identification accuracy of this paper’s model on the exercise risk (S2) is slightly lower than that of the BERT-CRF model, which is 0.03% lower. However, the accuracy rate of each other legal risk identification has achieved the highest results, and this paper’s model has good performance in the process of risk identification, verifying the effectiveness of this paper’s legal risk identification model in the identification of social work programs.