Study on the Digital Restoration of Costumes in the Picture of Palace Ladies with Silk Fans in Tang Dynasty

The Tang Dynasty was the heyday in Chinese history. Its costume culture reached unprecedented prosperity and became a peak in the history of Chinese costume. As a masterpiece of painting in this period, the picture of a lady with a fan provides us with valuable material for studying the costume culture of the Tang Dynasty.

Costumes in the Tang Dynasty are diverse and unique. In the Tang Dynasty, women's clothing was mainly skirts, with skirts as the top and skirts as the bottom dress. Combining the two formed a loose, natural dressing style [14, 15]. In addition, there are many styles, such as Hu clothes, men's clothes, half-arms, etc., which show the diverse pursuit of clothing by women in the Tang Dynasty. In The Picture of a Lady with a Fan, the costumes worn by ladies are rich in styles, which fully reflects the diversity of costumes in the Tang Dynasty. The use of colors is bold and full of the flavor of the times. Costumes in the Tang Dynasty are rich in bright red, green, blue, and other tones, as well as soft pink, purple, yellow, and other colors [16]. Women in the Tang Dynasty dared to try all kinds of color matching, which made their costumes show unique aesthetic effects. The ladies' costumes in The Picture of a Lady with a Fan are colorful and contrasted, which shows women's pursuit of a better life in the Tang Dynasty. Exquisite patterns and rich meanings. The costume patterns in the Tang Dynasty were mainly plants, animals, and geometric patterns, such as peonies, lotus flowers, butterflies, phoenixes, etc. [17]. These patterns have high aesthetic value and contain the beautiful meaning of auspiciousness and wealth. In The Picture of a Lady with a Fan, the patterns on the ladies' costumes are delicate and vivid, which reflects the unique charm of the costume patterns in the Tang Dynasty.

Tang Dynasty costumes are rich in materials and exquisite in production technology. High-grade fabrics such as silk, silk, and satin silk are widely used in clothing production, which makes clothing light, soft, and comfortable [18, 19]. At the same time, the costume-making technology in the Tang Dynasty was very particular, and embroidery, printing and dyeing, brocade, and other skills reached a very high level. The costume details in The Picture of a Lady with a Fan, such as the luster of silk threads and the exquisite patterns, all show the exquisite costume-making technology in the Tang Dynasty.

The purpose of image preprocessing is to highlight the prominence of the target and enhance the target information. In the process of clothing image extraction, due to the influence of fabric, texture, color, and other factors or the noise generated in image acquisition and transmission, it is necessary to carry out image filtering and noise reduction preprocessing [20, 21]. The processed image is more conducive to target extraction in the later stage. The image preprocessing technology analyzed in this section is mainly image filtering technology.

Image filtering can be divided into spatial and frequency domain filtering according to processing methods [22]. Filtering algorithms are classified into two types: linear and nonlinear filtering [23]. Currently, the commonly used methods include mean and Gaussian filtering [24]. Mean filtering calculates the arithmetic average, which works well in dealing with random noises like Gaussian and Poisson. The mean filter is given by equation (1): (1) g(x,y)=1mn∑ f(x+a,y+b) Where a, b∈Z;m×n is the size of filter; f(x, y) is pixel value; g(x, y) is output pixel. Mean filtering blurs image edges and causes ringing, unsuitable for clothing pattern filtering [25]. Gaussian filtering, a linear filter, averages pixel values with the highest weight on the target pixel, using a Gaussian function shape for weights [26]. It effectively handles normally distributed noise [27, 28]. Gaussian filter is presented in equations (2) - (3): (2) G(x,y)=12πσe−(x2+y2)2σ2 (3) g(x,y)=ΣG(x+a,y+b)⊗f(x,y) Where G(x, y) is weight of (x, y) pixel; σ is standard deviation; ⊗ is convolution operation. Nonlinear filtering, like linear filtering, has the output result determined by neighborhood pixels, but the former doesn't directly linearly combine related pixels and requires sorting the target pixels and neighborhood [29, 30]. The median filter is shown in Equation (4): (4) g(x,y)=med{f(s,t)} Where med represents the median function. (s, t) are all pixel coordinates within the template; f(s, t) are all neighborhood pixel values including target pixel; g(x, y) is output pixel value.

Although median filtering features an excellent image smoothing effect and clear edge preservation, when the pixel width of the detailed part of the image is less than half the size of the filtering template, median filtering will remove this part of the pixel value, which readily leads to detail loss and smoothes sharp corners. Adaptive median filtering addresses this issue, with the size of the filtering window being adjustable as needed. If target pixel value to be processed lies between maximum and minimum pixel values in the corresponding template neighborhood, the target pixel value is directly output as the filtered result value; otherwise, the median value after sorting the neighborhood pixels (including the target) is output, thus avoiding the over-smoothing problem of traditional median filtering while filtering noise.

To reduce noise before contour extraction, traditional filtering algorithms (mean, Gaussian, median) are often used in clothing pattern processing. However, they blur edges, causing detail distortion. For images like "The Lady with a Fan," where patterns are distinguished by blank space, these algorithms blur edges and lose details, hindering subsequent segmentation and contour extraction. The relative total variation model (RTV) model uses a unique algorithm to differentiate noise from stable structures, preserving edges while smoothing noise.

After the image is preprocessed, it's necessary to segment and extract the object. Research has shown that the most commonly used method currently is to cluster the single target image as a whole to achieve automatic pattern segmentation. Although this method can accurately obtain the segmentation results of patterns, this kind of integrated image color clustering method can't accurately separate the background from the target pattern. For the image of the "Lady with a Fan" costume, which is produced by a unique carving process and closely combined with the shadow's torso, with a simple outer outline but concentrated internal patterns and a relatively complex background, the independence and integrity of the extracted patterns will be lacking. Based on Graph cuts, the Grabcut interactive pattern segmentation method proposed by Rother et al. uses the Gauss hybrid model instead of the gray histogram model to model the foreground and background separately and segments the foreground and background through simple interactive operations. This method's segmentation is inaccurate due to similar colors between background and target. The Gauss hybrid model also slows the algorithm. This study combines simple linear iterative clustering algorithm (SLIC) and GrabCut for Lady Waving Fan costume images, refining segmentation and using GrabCut for accurate, efficient target extraction.

The process of GABRS, the clothing pattern segmentation algorithm designed in this study, is shown in Figure 1: first, smooth the input clothing image of Lady Waving Fan; second, use a simple linear iterative clustering algorithm to perform super-pixel segmentation on the smoothed image; finally, use the GrabCut algorithm to segment and extract local patterns.

Figure 1.

Flowchart of image segmentation algorithm

Because the production of the filmmaker in the costume art of "A Lady Waving a Fan" uses artificial coloring treatment after cutting animal skins, such as donkey skins, stains and uneven coloring can be generated due to technical reasons or coloring processes, and these randomly generated stains (noises) are often irregular, it is difficult to extract the clear outline of the costume pattern of Lady Waving Fan. Due to its unique variation algorithm, the relative total variation model (RTV) can clearly differentiate the irregular (changeable direction) texture from the relatively stable main structure and effectively isolate the noise texture from the main structure outline in the costume image of "The Lady with a Fan", possessing strong texture suppression robustness. The algorithm first computes the relative total variation V of the image I=(I₁,I₂,...,I_N) and then minimizes it by parameters argmin ∑_p(S_p – I_p)² + V to ensure that the input image I and the output image S=(S₁,S₂,...,S_N) have a similar main structure. While performing denoising, the model effectively preserves the main structure outline of the costume image in "The Lady with a Fan". In this paper, the RTV algorithm is employed to smooth the costume image of "Lady Waving Fan". The specific steps are detailed as follows:

Calculate the relative total variation V=Dx(p)Lx(p)+ε+Dy(p)Ly(p)+ε in the square neighborhood R (p) centered on p, the total window variations D_x (p), D_y (p) in the x and y directions are equations (5)-(6): (5) Dx(p)=∑q∈R(p)gp,q| (∂xS)q | (6) Dy(p)=∑q∈R(p)gp,q| (∂yS)q |

The intrinsic variations L_x (p), L_y (p) within the window are equations (7)-(8): (7) Lx(p)=| ∑q∈R(p)gp,q(∂xS)q | (8) Ly(p)=| ∑q∈R(p)gp,q(∂yS)q | Where q is the index of all pixel points within the R(p) field, ∂_x, ∂_y is the incomplete differential of pixel q in x and y directions respectively. g_p,g is the weight functions that define the spatial relationship, as shown in equation (9): (9) gp,q∝exp[ −(xp−xq)2+(yp−yq)22σ2 ] Where σ is the spatial scale parameter that controls the size of the window. It is selected based on the size of the processed image texture, which determines the smoothness of the image. Minimize the parameters and calculate the image S after filtering out the small-scale texture, as presented in Equation (10). (10) argmin∑p(Sp−Ip)2+λ·(Dx(p)Lx(p)+ε+Dy(p)Ly(p)+ε) Where λ is the image smoothness coefficient, ε is the parameter deviation value, both of which concurrently govern the ratio of the fidelity term to the relative total variation and also regulate the smoothness of the image.

To enhance the efficiency and accuracy of image segmentation using the GrabCut algorithm and decrease the algorithm's complexity, this paper initially employs the SLIC algorithm to extract super pixel blocks prior to segmenting the costume image of Lady with a Fan. The SLIC algorithm is a clustering approach that creates super pixel blocks for image segmentation based on the closeness of color difference and distance. It is appropriate for clothing images of "Lady with Fan" that have high color saturation and a notable difference between color blocks.

First, specify the number n of superpixels to be generated, and then transfer the image to the CIELab space. The three-channel value (L, a, b) corresponding to each pixel and the point coordinate (x, y) constitute a vector C=[L,a,b,x,y]^T. The visualize S includes N pixel points, and the distance between clustering center points (seed points) of adjacent superpixel blocks is H. Within the 2H * 2H domain, the similarity (distance) of two pixels is measured by their color difference d_Lab and positional distance d_xy. n cluster center vectors are C_j=[L_j,a_j,b_j,x_j,y_j,]^T,j∈{1,2,.,.,n}. The distance D_s from the point C_i=[L_i,a_i,b_i,x_i,y_i]^T,i∈{1,2,...,N} to the cluster center is defined as equations (11)-(13): (11) dLab=(Lj−Li)2+(aj−ai)2+(bj−bi)2 (12) dxy=(xj−xi)2+(yj−yi)2 (13) Ds=dLab2+(dxyH)2m2 m is a parameter that controls the spatial tightness of superpixels, and the value range is 1 ~ 20 (the default is 1). The larger m is, the more emphasis is placed on spatial proximity, and the more regular the shape of superpixels becomes.

In this study, adaptive curve fitting method is proposed. The defective part of the structural line is reconstructed according to line trend, and inverse distance weighted interpolation method is used for coloring to make the reconstructed structural line more realistic. Bezier curve is widely used in drawing pattern outlines and is an effective method to repair defect structures in silk cultural relics images. This paper proposes an adaptive curve fitting algorithm based on quadratic Bezier curves to construct missing structural lines.

The selection of control points determines the applicability of the fitted curve. Two selection methods are provided. If control points are known, their coordinates can be directly input. If control points are unknown, this paper designs a quadratic Bezier curve fitting method for adaptive control points and uses slope formula to determine them. Calculate slope according to horizontal and vertical coordinates of starting and ending points of the curve, and then use the slope to get tangents at these two points. Take intersection of the two tangent lines as control point of quadratic Bezier curve fitting and participate in the calculation. Finally, construct missing edges using starting point, control point, and endpoint until all missing structural lines are reconstructed.

Color image can be represented as matrix (m × n × 3), where m and n are width and height of the image respectively. Three-channel values of a single pixel can be described as red, green, and blue values. Pixels at both ends of structure line to be fitted are selected interactively. Pattern should be restored to its original shape as much as possible. Fitted structural lines should be the same color as the original pattern. Inverse distance weighted interpolation method is used to keep the color of fitted structure line consistent with original structure line. For incomplete portion of the pattern, pattern structure line can be fitted according to the connection trend of pattern structure line, thereby dividing the defective region of the image into a plurality of regions that need to be repaired.

The restoration of silk artifacts images relies heavily on image information outside the defective area. The image restoration technology architecture is shown in Figure 2. The defect area to be repaired is mainly divided into the structural part and the content part. Based on the example image restoration algorithm, this paper realizes the restoration of silk cultural relics images. Sobel edge detection operator is used to detect edge of the area to be repaired. Boundary function B(p) is introduced to improve boundary priority of the filling leading edge. Improving confidence C(p) of the later repaired blocks. The credibility of the restored image is improved.

Figure 2.

Image restoration technology architecture

The setting of sample block size has a certain influence on the algorithm's repair results. If sample block size is too small, it contains less information and requires time to enhance the repair. Instead, subtle texture information will be lost. The sparsity value reflects the characteristics of the sample block. Smaller sample blocks are selected to maintain structural information, larger sample blocks are selected to maintain the integrity of texture features, and fine texture information is maintained without losing structural information so as to avoid structural line disconnection and block effect. To sum up, the above analysis, if the structural sparsity value corresponding to the sample block is relatively large, the sample block with a smaller size should be selected, and vice versa. This method initializes the size of the sample block prior to the repair process by employing an adaptive function.