Tobacco adulteration recognition study by hyperspectral data processing under machine learning

The advent of the era of artificial intelligence and big data, so that human society is undergoing an unprecedented change, especially in the industrial performance is particularly prominent, such as the defect detection of the bottle cap, this use of the camera to capture the target image based on the computer vision solution has a high degree of automation, high precision and stability, etc., to improve the production efficiency of the enterprise at the same time, but also to increase the economic benefits of the enterprise, and more importantly, improves market competitiveness [1-2]. In addition, the use of computer vision to solve these problems is the ultimate goal is to improve product quality, to ensure that qualified products are delivered to the user or customer, if the products mixed with debris into the market, then the consumer's health will pose a serious threat to the credibility of the enterprise will be affected, so to ensure that the purity of the raw material is particularly important [3-4].

Tobacco industry, like food, pharmaceutical industry, the purity of raw materials is also very strict control, deterioration or adulteration of raw materials will undoubtedly have a serious impact on the quality of the finished product, the enterprise in the production process of raw materials of foreign matter rejection occupies a very important part of the [5-6]. Early foreign matter rejection is taken in an artificial way, by arranging labor workers in the production line with the naked eye to observe and manually pick out the foreign matter in the tobacco, this method is simple but improve the production cost, more importantly, the effect of manual rejection is often unstable, depending on the subjective decision-making of the labor workers, the accuracy of the lower, nowadays more use of intelligent tobacco foreign matter rejection machine, the equipment is a set of optical, electrical, This equipment is a collection of optical, electrical, computer, mechanical in one of the automated rejection equipment, the use of CCD high-speed camera to collect images of the tobacco stream, and then transmitted to the industrial computer for image analysis and understanding to identify the location of the foreign body and transmitted to the rejection valve, the rejection valve using high-speed downward pressure airflow will be removed from the foreign body [7-9]. The core part of the device is the tobacco foreign matter recognition algorithm, there are two types of recognition algorithms, one is based on the color recognition algorithm, the other is based on the texture recognition algorithm, color-based recognition algorithms due to its speed and is widely used, and in the color recognition algorithms how to establish a perfect color model for the color of the tobacco leaf is the key to this type of algorithm [10-12].

In the tobacco online foreign matter rejection of this link, North America, Western Europe and Japan, some of the larger tobacco companies are using optical debris automatic sorting equipment, they have become an indispensable general equipment in the silk production line. In China, there are very few cigarette factories have introduced foreign optical tobacco sorting equipment Tobacco Sorter, but it is expensive about 800,000 U.S. dollars per unit, expensive spare parts, after-sales maintenance services are difficult to get in place in a timely manner, and more importantly, by the intellectual property rights of the products are subject to the constraints of the introduction of the same level of equipment with foreign countries, and it is difficult to increase the competitiveness of the enterprise's international market [13-14]. Therefore, the development of tobacco online foreign matter rejection system with independent intellectual property rights is of great significance to save the expenditure of tobacco enterprises, stabilize the intrinsic quality of tobacco products, solidify and develop the famous cigarette brands, improve the automation and control level of silk production line, and enhance the competitiveness of China's tobacco enterprises in the international tobacco market, etc. [15].

In this paper, the region of interest of the image data of tobacco samples was selected to distinguish the samples from the environment and reduce the interference caused by the background information. The SNV method was adopted to preprocess the hyperspectral data to correct the hyperspectral error caused by the scattering effect, followed by PCA to extract features from tobacco data of different qualities, and to transform the multi-dimensional bands in the original data into a low-dimensional sample matrix, so as to maximize the retention of the key data in the original data. GoogLeNet was adopted as the pre-training model for deep learning, and certain non-essential layers of GoogLeNet were removed and new layers were added, in addition to changing the activation function of the model, to achieve the purpose of improving the CNN recognition model, and to improve its performance in tobacco adulteration experiments.

In order to improve the training performance, GoogLeNet was used in this study as a deep learning pre-training model, which was used as a basis to train the classifier. In the improved CNN recognition network classifier, the last 5 layers of GoogLeNet were removed and 15 new layers were added. In addition, the ReLU activation function in the feature map layer is changed to a Leaky Rectified Linear Unit (LeakyReLU) activation function [17] to improve the expressiveness of the model and to overcome the problem of vanishing ReLU gradients without disturbing the primary convolutional neural network architecture. The structure of the improved CNN recognition network is shown in Fig. 1 [18].

Figure 1.

Improved CNN recognition network structure

Where Conv_i denotes the convolutional layer, i=1, 2, 3, 4; BN denotes the batch normalization layer. LReLU denotes the activation function of LeakyReIU. Apool denotes the average pooling layer. FC denotes the fully connected layer. The first convolutional layer (Conv_l) in the improved CNN recognition network uses a 1 × 1 sized filter, thus effectively reducing the size of the image. The second convolutional layer (Conv_2) uses a 3 × 3 convolutional block, thus having a dimension reduction effect. In addition, the starting module of GoogLeNet has different convolutional kernels such as 1 × 1, 3 × 3 and 5 × 5 convolutional kernels for extracting features at different scales.

In general, larger convolutional kernels will cover a larger area to compute features. 1 × 1 convolutional kernels on the other hand provide more detailed features and reduce the amount of computation. The new additions in the improved CNN recognition network include four convolutional layers, all with filter sizes of 1 × 1. In addition, the increased number of convolutional layers in the CNN facilitates the provision of more detailed, accurate, and robust features. The four newly added convolutional layers allow for the extraction of higher level features compared to the initial layer that extracts lower level features.

Convolutional layers are used to learn deep features from the input image and generate a feature map. Improved CNN uses different sized filters in the convolutional layers and 3 × 3 filters in the maximum pooling layer. The discrete convolution computation process is shown in equation (3): (3) s(t)=(x*w)(t)=∑a=−∞∞x(a)w(t−a) \[s(t)=(x*w)(t)=\sum\limits_{a=-\infty }^{\infty }{x}(a)w(t-a)\]

Where: w is the filter, x is the input feature, t is the time variable and s is the output feature.

Activation functions are usually used in deep learning based models for nonlinear transformation processes. In this study, LeakyReLU activation function is used instead of the traditional ReLU to solve the ReLU gradient vanishing problem.The equation of LeakyReLU activation function is as follows: (4) f(x)={ xx≥0ζxx<0 \[f(x)=\left\{ \begin{array}{*{35}{l}} x & x\ge 0 \\ \zeta x & x<0 \\ \end{array} \right.\]

Where: ζ is a tiny positive number for outputting a small value in case of negative values, in this study ζ is taken to be 0. 01. Thus, no neuron is deactivated in this case and thus neuron activity can be maintained without loss of function (outputting 0 results in neuron failure).

This study also utilizes the batch normalization layer to perform normalization operations on the output generated by the convolutional layer. Normalization effectively reduces the training time of the model for a more efficient and faster learning process. The batch normalization process is calculated as follows: (5) { yi=xi−μσ2+εμ=1M∑i=1Mxiσ=1M(xi−μ)2 \[\left\{ \begin{array}{*{35}{l}} {{y}_{i}}=\frac{{{x}_{i}}-\mu }{\sqrt{{{\sigma }^{2}}+\varepsilon }} \\ \mu =\frac{1}{M}\sum\limits_{i=1}^{M}{{{x}_{i}}} \\ \sigma =\frac{1}{M}{{({{x}_{i}}-\mu )}^{2}} \\ \end{array} \right.\]

Where: y_i is the ind batch normalized value, M is the total number of input data, μ is the data mean, and σ is the standard deviation.

After the convolutional layer, a pooling layer is used to simplify the information coming from the convolutional layer (a downsampling process to reduce the size of the feature map and remove unnecessary data). Average pooling and maximum pooling are the 2 most common pooling strategies. In the last 15 layers, this study uses a global average pooling operation. In the pooling layer, the network does not perform any learning. The pooling process in this study uses a 3 × 3 sized filter. The pooling process is as follows: (6) { s=w2h2d2w2=(w1−f)A+1h2=(h1−f)A+1d2=d1 \[\left\{ \begin{matrix} s={{w}_{2}}{{h}_{2}}{{d}_{2}} \\ {{w}_{2}}=\frac{({{w}_{1}}-f)}{A+1} \\ {{h}_{2}}=\frac{({{h}_{1}}-f)}{A+1} \\ {{d}_{2}}={{d}_{1}} \\ \end{matrix} \right.\]

Where: w₁ is the width of the input feature map, h₁ is the height of the input feature image, d₁ is the depth value of the input feature image, f is the filter size, A is the number of steps used, and s is the size of the output feature map after pooling operation.

In the improved CNN, the convolutional layer is followed by a fully connected layer, and this part of the network structure is responsible for mapping the high-level features extracted from the convolutional layer to the final tobacco adulteration recognition result. Specifically, the convolutional layer extracts features from the input data through a series of filters, which may then undergo dimensionality reduction through a pooling layer to reduce the number of parameters and improve invariance to image displacement. Next, these features are further nonlinearly transformed through one or more fully connected layers, culminating in an output layer that determines the probability of each category. The fully connected layers determine the most significant patterns in order to categorize the image. The fully connected layer is computed as follows: (7) { uil=∑jωjil−1yjl−1yil=f( uil )+bl \[\left\{ \begin{array}{*{35}{l}} u_{i}^{l}=\sum\limits_{j}{\omega _{ji}^{l-1}}y_{j}^{l-1} \\ y_{i}^{l}=f(\;u_{i}^{l}\;)+{{b}^{l}} \\ \end{array} \right.\]

Where: l is the total number of layers of the network, i and j are the total number of neurons, yil$y_{i}^{l}$ is the value of the output of the fully connected layer, ωkl−1$\omega _{k}^{l}-1$ is the value of the weights of the hidden layer, yjl−1$y_{j}^{l}-1$ is the value of the input neurons, uil$u_{i}^{l}$ is the value of the output layer; b^l is the bias.

The activation function in the improved CNN makes the output of the fully connected layer more normalized and the Softmax function calculates the probability for each tobacco category and outputs the final tobacco category to derive its adulteration result.The formula for calculating the probability of the Softmax function is as follows: (8) P(y=j | xi,ω,b)=exTωj∑j=1nexTωj \[P(y=j\;|\;{{x}_{i}},\omega ,b)=\frac{{{e}^{{{x}^{T}}{{\omega }_{j}}}}}{\sum\limits_{j=1}^{n}{{{e}^{{{x}^{T}}{{\omega }_{j}}}}}}\]

Where: P is the input to the Softmax function computation x_i is the probability of category j, ω and b are the parameter weight vectors and bias vectors, respectively, and n is the number of categories.

Tobacco market often appear as a good, inferior tobacco mixed into the top grade tobacco mixed sales, bad tobacco traders to seek high profits, so this paper designed a tobacco adulteration identification experiment. Tobacco for testing (provided by a tobacco company) types of tobacco, respectively, superior tobacco - mature leaf level 1, medium tobacco - smooth leaf level 1, inferior tobacco - green and yellow smoke level 1, each variety were collected 200 samples. The samples were divided into a calibration set and a test set according to the ratio of 3:1. The calibration set was used for training validation and model parameter optimization, and the test set was used for evaluating the performance of the model on unknown samples.

4.2

Visualization and Preprocessing of Raw Spectral Data

The extracted raw spectral data were visualized using the PCA algorithm, which can reflect the main characteristics of different tobaccos, and the results of the characteristic visualization are shown in Fig. 2, with the black circle being the superior tobacco-perfected leaf level 1, the blue triangle being the medium tobacco-smooth leaf level 1, and the orange pentagon being the inferior tobacco-green and yellow tobacco level 1. The three types of tobacco have relatively independent spatial distribution, which can visualize that these three types of tobacco are classifiable and have strong theoretical guidance for subsequent classification.

Figure 2.

Visual results of tobacco pca

The spectral data can be detected by the LOF algorithm to find an anomalous sample data, which is eliminated and the remaining sample data is used for subsequent data processing. After the elimination of the sample spectral data with MATLAB R2018a for graphing. Because too many curves are concentrated in the graph, it is not easy to see the difference between the reflectance values of the three types of tea spectral data, so all the sample spectral data of each type of tea were averaged and then plotted, and the average value of the spectral data is shown in Figure 3. From Figure 3, it can be seen that the spectral curve trend of these three types of tobacco is probably similar, with three peaks and one trough, and due to the differences in the internal structure of the three types of tobacco, there are obvious differences in the reflectance values (fluorescence intensity) at the three peaks and one trough, and the three types of tobacco have the greatest differences in fluorescence intensity between 720 nm-780 nm, the lowest fluorescence intensity of the finished leaves, and the highest fluorescence intensity of the green and yellow tobaccos. These three peaks are near 510nm, 700nm and 750nm respectively, and the trough is near 730nm, from which it can be inferred that the bands near these four bands will be used as the characteristic bands and play a dominant role in the identification of the three types of tobacco adulteration.

Figure 3.

The average spectral curve of three kinds of tobacco

When these data were collected, the spectral data collected from different samples varied because of the noise data caused by light stability, instrument stability, dark current, light scattering, light refraction, and other light shifts when the samples to be tested were photographed. In order to eliminate the influence of non-quality factor information in the fluorescence hyperspectral data, and to obtain the hyperspectral data with original features as the input set for subsequent modeling, the sample spectral data were preprocessed using the PCV algorithm, and in addition, the SG and GWS preprocessing methods were used for a comparative analysis with the preprocessing method in this paper. After preprocessing, the recognition is carried out with the models SVM, RF and KNN and the improved CNN model in this paper, respectively, and the recognition accuracy is shown in Fig. 4, and the modeling time is shown in Table 4.

Figure 4.

Different pretreatment method classification effect

Table 4.

Modeling time

Methods	Modeling Time(s)
	SVM	RF	KNN	This model
RAW	0.06	0.04	0.012	0.0003
SG	0.06	0.04	0.012	0.0003
GWS	0.062	0.043	0.011	0.00032
PCA	0.057	0.038	0.009	0.00021

From Table 4 and Fig. 4, it can be seen that: the classification effect of model SVM is the worst, the original data without data preprocessing is only 77.45%, and the original data after SG preprocessing has no effect on the classification effect, and after GWS preprocessing the classification effect is reduced instead, only 75.49%, and the classification effect of this model is much lower than the other three.

The classification effect of the model in this paper is the best, the original data without data preprocessing reached 92.15%, after the SG preprocessing has little effect on the classification effect, after the GWS preprocessing the classification effect is reduced instead, only 90.19%, after the PCA processed the recognition accuracy is significantly improved, reached 97.56%.

While the original data of this paper's model is not pre-processed modeling time is 0.0003s, after SG pre-processing modeling time is 0.0003s, there is no enhancement, after GWS pre-processing modeling time is 0.00032s, not enhancement instead of reduction, after this paper's pre-processing method, the modeling time is 0.00021s, which is enhanced by 30%.

From the above analysis, it can be seen that the application of PCA preprocessing method for feature dimensionality reduction and then use the improved CNN model for tobacco adulteration identification has the highest accuracy and reduces the modeling time most effectively.

In this paper, a method based on hyperspectral image technology is proposed to accurately detect the type of tobacco adulteration and the origin of adulteration, which shows good classification and recognition effect in detecting the adulteration of tobacco.

The recognition effect of the improved model in this paper is the best, the recognition accuracy of the original data reaches 92.15%, and the recognition accuracy after PCA processing is further improved to 97.56%. The modeling time of the original data of this paper's improved model without preprocessing is only 0.0003s, and after the original data dimensionality reduction by this paper's preprocessing method, the modeling time is reduced by 30% to 0.00021s, which illustrates that the tobacco adulteration recognition scheme combining the PCA preprocessing method and the improved CNN model of this paper has the best performance in terms of both accuracy and efficiency.

The KNN model, RF model and the improved CNN model in this paper all achieve better recognition results in tobacco adulteration origin recognition, and the recognition accuracy of the three models is higher than that of manual recognition, in which the improved model in this paper performs particularly well, with the recognition accuracy, precision, and recall reaching 98.28%, 98.87%, and 98.94%, respectively, and the F1 value is 97.88%. It proves the superior effect of the improved CNN model network structure in this paper. It can play the best performance in the problem of tobacco adulteration origin recognition, distinguishing the tobacco with different quality from different origins, and contributing to the research of tobacco adulteration recognition.

China Tobacco Hebei Industrial Co., LTD., intelligent research and application technology project of finished tobacco purity based on hyperspectral technology (HBZY2023A084).