Integration and Innovation of Traditional Ceramic Art and Modern Film and Television Scene Designs

Figure 1 shows the schematic of pinhole camera model imaging, 3D reconstruction is to restore the representation of an object in 3D space by image information from multiple viewpoints [28]. And the camera imaging principle describes how the camera projects real-world objects onto a two-dimensional image. The pinhole camera model, as a classical simplified model, explains the imaging principle of the camera, which involves the transformation relationship between the world coordinate system, the camera coordinate system, the image coordinate system and the pixel coordinates.

Figure 1.

Pinhole camera model imaging schematic

World Coordinate System: A coordinate system used to represent the position of an object in the real world, usually represented by O_W − X_WY_WZ_W. Where O_W is a point in real space. For example, point P has coordinates (XW,YW,ZW) in the world coordinate system.

Camera coordinate system: the camera coordinate system is a coordinate system defined relative to the camera itself, usually expressed as O_C − X_CY_CZ_C. Where O_C represents the optical center of the camera, i.e. the center of the point of light. The optical axis of the camera is defined as the Z-axis, which is usually used to describe the direction of view of the camera. Point P has coordinates (XC,YC,ZC) in the camera coordinate system.

Image coordinate system: According to the imaging principle of the pinhole camera, the imaging plane is located behind the camera plane. Take the intersection point O of axis Z of the camera coordinate system and the imaging plane as the origin to establish the image coordinate system O − xy, and the coordinate of point P in the image coordinate system is (x,y) .

Pixel coordinate system: Establish the pixel coordinate system O_θ−uv with the vertex O_θ of the captured image as the origin, and the coordinate of point P in the pixel coordinate system is (u,v) . 1)

Conversion from the world coordinate system to the camera coordinate system

During the conversion from the world coordinate system to the camera coordinate system, no deformation of the object occurs, only the coordinate system is converted. This conversion is accomplished by translation vector t and rotation matrix R, as shown in equation (1): (1)[ XC YC ZC]=R[ XW YW ZW]+t

where R is a 3 × 3-unit orthogonal matrix that describes the rotation transformation of the world coordinate system with respect to the camera coordinate system. t=[tx,ty,tz]T is the offset between the world coordinate system origin O_W and the camera coordinate system origin O_C. To simplify the computation, the rotation matrix R and the translation vector t are combined into one chi-square transformation matrix. This allows the points of the world coordinate system to be transformed into the camera coordinate system using matrix operations as shown in equation (2): (2)[ XC YC ZC 1]=[ R t 0T 1][ XW YW ZW 1]

2)

Conversion from camera coordinate system to image coordinate system

The process of converting from a camera coordinate system to an image coordinate system is a process of projecting three-dimensional points onto a two-dimensional plane. This conversion through the camera’s optical center of the three-dimensional point P projected onto the imaging plane, forming a similar triangle relationship. By the principle of similar triangle, the point P in the camera coordinate system can be converted to the coordinates in the image coordinate system.

Based on the properties of similar triangles, the following formula can be obtained: (3){ Zcf=Xcx=Ycy x=fXcZc,y=fYcZc,z=f}

where f denotes the focal length, which is the distance between the camera coordinate system and the origin of the image coordinate system.

According to the derivation, the chi-square transformation formula from the camera coordinate system point P(XC,YC,ZC) to the image coordinate system point P(x, y) is: (4)[ x y 1]=1Zc[ f 0 0 0 0 f 0 0 0 0 1 0][ XC YC ZC 1]

3)

Conversion from image coordinate system to pixel coordinate system

In image processing, the pixel coordinate system is usually used to describe the position of pixels, so after completing the conversion from the camera coordinate system to the image coordinate system, it is necessary to convert the image coordinate system to the pixel coordinate system. The conversion between the pixel coordinate system and the image coordinate system includes scaling and translation of the origin. Let the corresponding real physical dimensions of a pixel point on the imaging plane be dx and dy, and the difference between the origin of the two coordinate systems in the direction of the horizontal and vertical axes be u₀ and v₀, respectively, then the correspondence between them can be expressed as follows: (5){ u=xdx+u0 v=ydy+v0}

Collapsed into the chi-square conversion equation is: (6)[ u v 1]=[ 1dx 0 u0 0 1dy v0 0 0 1][ x y 1]

By combining Eqs. (2), (4) and (6), the conversion equation from the world coordinate system to the pixel coordinate system can be derived as: (7)[ u v 1]=1Zc[ 1dx 0 u0 0 1dy v0 0 0 1][ f 0 0 0 0 f 0 0 0 0 1 0][ R t 0T 1][ XW YW ZW 1]=KTZc[ XW YW ZW 1]

Order: (8)K=[ 1dx 0 u0 0 1dy v0 0 0 1][ f 0 0 0 0 f 0 0 0 0 1 0],T=[ R t 0T 1]

Where K represents the internal reference matrix of the camera, which is usually a fixed 3 × 3 matrix given by the camera factory and contains internal parameters such as the camera’s focal length, principal point offset, etc. T represents the external reference matrix of the camera, which includes the rotation matrix R and translation vector t. The external reference matrix changes with the position and orientation of the camera in space and is used to describe the pose and position of the camera relative to the world coordinate system. Z_C represents the depth value of the point in the camera coordinate system, i.e. the distance between the point and the optical center of the camera.

Figure 2 shows the schematic of the pair-pole geometric constraints, when it is known that there are a certain number of feature matching points between two or more images, the inference of the camera parameters and the determination of the depth range can be accomplished according to the principle of the pair-pole geometric constraints. This process utilizes the geometric relationship between the cameras, and through the analysis and processing of the matching points, the internal and external parameters of the camera as well as the depth information in the scene can be inferred.

Figure 2.

For polar geometry constraints

In Fig. 2, points O₁ and O₂ represent the optical centers of the two cameras, respectively. Point P is a point in three-dimensional space, and p₁ and p₂ are the projection points of point P on the imaging planes I₁ and I₂ of the two cameras, respectively. Points e₁ and e₂ are the intersections of the optical centers O₁ and O₂ with the two imaging planes. The plane formed by points O₁, O₂ and P is called the polar plane. The lines of intersection of the imaging planes I₁ and I₂ with the polar planes p₁e₁ and p₂e₂ are called polar lines. As can be seen from the figure, the projection points of all points on O₁P on the imaging plane I₂ can only fall on the polar line p₂e₂. From Eq. (7), Eq. (9) can be obtained to express the corresponding coordinate relationship: (9){ s1p1=KP s2p2=KT12P=K(R12P+t12)}

where s_l and s₂ are the depths of point P taken in the O_l and O₂ camera coordinate systems, respectively, and the rotation matrix for transforming the O_l camera coordinate system to the O₂ camera coordinate system is R₁₂, and the translation vector is t₁₂. This can be obtained by combining in Eq. (9): (10)s2p2=K(R12K−1s1p1+t12)

Multiply both sides by K⁻¹ at the same time to get: (11)K−1s2p2=R12K−1s1p1+t12

Let x₂ = K⁻¹s₂p₂, x₁ = K⁻¹s₁p₁, be obtained: (12)x2=R12x1+t12

Simultaneously making an outer product with t₁₂ for both sides of the equation, denoted by t₁₂, and simultaneously left-multiplying by x₂^T, the following simplified equation is obtained since the inner product of the vertical vectors is zero: (13)x2Tt12∧R12x1=0

Substituting x₂ and x_I gives: (14)p2TK−Tt12ΛR12K−1p1=0

Noting the middle part of this expression as basis matrix F and essence matrix E, respectively, one can simplify the pairwise polar geometric constraints to: (15)x2TEx1=p2TFp1=0,E=t12ΛR12,F=K−TEK−1

In the case where the internal and external parameters of the camera are unknown at the same time, the classical 8-point method can usually be used, i.e., the basis matrix F is solved based on 8 pairs of matching points.

After depth estimation is complete, filtering out outliers from the background and occluded regions is a critical step to ensure the quality and accuracy of the final point cloud [29]. The depth map is filtered by the geometric consistency between multiple images to eliminate the depth values that do not fulfill the conditions. A pixel p exists on the reference depth map I₀ with depth dp. A pixel p is projected to a pixel q in another depth map I_l with depth dq according to its depth value dp. A pixel q is then projected back to the reference depth map I₀ according to its depth value dq. If the reprojected coordinate d_reproj and the reprojected depth d_reproj satisfy Equation (16), the depth estimate at p is said to be satisfying the geometric consistency of the two-views. In the experiments in this paper, ensuring that the depth estimate at a point is not rejected requires that at least three-view consistency is satisfied: (16){ |prepvoj−p|<1 |drepvoj−dp|dp<0.01}

After the depth map filtering of all the images, the depth map fusion algorithm converts each depth map to a unified coordinate system, generating a 3D point cloud model of the scene and realizing the 3D reconstruction of the target. This paper uses the open source fusibile code to realize this function.

After obtaining the depth map fused point cloud, there are two more steps before the complete 3D model, Mesh reconstruction is the process of generating a continuous 3D mesh model from discrete point cloud data. Usually, after obtaining a depth map fused point cloud, there are two more steps before a complete 3D model is available. Mesh reconstruction is the process of generating a continuous 3D mesh model from discrete point cloud data.

In this paper, we use UV mapping to realize this process. UV mapping is a special way of mapping texture coordinates, where UV coordinates represent positions on a 2D texture image, similar to X and Y coordinates. In UV mapping, each vertex is assigned a UV coordinate that is used to determine the position of that vertex on the texture image. The range of the UV coordinates is usually [0,1], where (0,0) denotes the lower-left corner of the texture image and (1,1) denotes the upper-right corner. By mapping the UV coordinates to the texture image, it is possible to realize the texture mapping to precisely fit the surface of the 3D model, thus giving the model a more vivid appearance.

Several mathematical formulas and technical descriptions are used in the calculation of hand positions for the Leap Motion device. These formulas cover areas such as computer vision, coordinate transformations, and geometric calculations. Leap Motion uses dual cameras to capture hand positions, so points in the camera coordinate system need to be transformed to the world coordinate system.

Point (xc,yc,zc) in the camera coordinate system can be converted to point (xw,yw,zw) in the world coordinate system by the projection matrix P of the camera as shown in equation (17): (17)( xw yw zw 1)=P( xc yc zc 1)

The projection matrix P can usually be expressed as equation (18): (18)P=K[R|t]

In Equation (18), K is the camera’s internal reference matrix, R is the rotation matrix, and t is the translation vector.

Leap Motion uses stereo vision to determine the depth information through the parallax of the two cameras. Assuming that there are two cameras with parallel optical axes and the baseline distance between the cameras is B, the depth z can be computed for the corresponding points (xL,yL) and (xR,yR) of the left and right cameras, as shown in Equation (19): (19)z=f⋅BxL−xR

In Equation (19), f is the focal length of the camera and x_L − x_R is the horizontal parallax of the corresponding points of the two cameras. With the depth value z calculated from the parallax, points (x, y) in the camera plane can be converted to points (X, Y, Z) in 3D space to realize 3D reconstruction as shown in Equation (20): (20)X=x⋅zf,Y=y⋅zf,Z=z

In addition to position information and 3D reconstruction, Leap Motion can recognize the posture and movement trajectory of the finger. Assuming that the 3D position of the finger joint is pi=(xi,yi,zi) , the orientation vector d_i of the finger bones can be expressed as Equation (21): (21)di=pi+1−pi

In order to recognize different gestures, the angle between the finger bone direction vectors can be calculated. Assume that the angle θ between the two vectors d₁ and d₂ is computed by Eq. (22): (22)cosθ=d1⋅d2‖d1‖‖d2‖

In Equation (22), · denotes the dot product and ‖d‖ denotes the mode of the vector.

Based on the above gesture tracking model with respect to scattered hand data features, the computational formula is specified as: (23)Di=‖Fi−C‖S,i=1,⋯,5

where the Euclidean distance between the tip of the finger and the palm of the hand is described by D_i and S represents the scale factor, which is obtained by the calculation of Eq. (24): (24)S=‖Fmiddle−C‖

The normalization process (except for the direction) of all hand eigenvalues is done by dividing the three-dimensional distance between the tip of the finger and the palm of the hand by the scale factor S.

The angle between the 3D coordinate points Fip and C and h in the commercial display space is A_i, which is calculated by the formula: (25)Ai=∠(Fip−C,h)

where Fip represents the projected position of the finger tip on vector n. The normalized vertical distance of the finger tip from the center of the palm is described by E_i, which is calculated as: (26)Ei=sgn((Fi−Fip)⋅n)‖Fi−Fip‖S

Each gesture data extracted by Leap Motion will be characterized and will comprise a set of eigenvectors V=(D,A,E) including eigenvalues D, A, and E eigenvalues.

DenseNet is a common convolutional neural network [31]. The basic idea of DenseNet continues ResNet, and the core features are all utilizing the residual network structure. The DenseNet network contains a number of dense blocks, in which the size of the feature maps does not generally change in size. In order to prevent the poor effect of the downsampling layer, downsampling layer is added between each dense block to reduce the feature dimension. Defining x₀ as the first layer of a convolutional neural network, l denotes the layer, and H_l(·) is the composite transform function of the layer, then the general convolutional output of layer l + 1 is x_l as shown in Eq. (27). ResNet uses cross-layer connections as in equation (28), i.e., it uses the output x_l−1 of layer l plus its transformed result Hl(xl−1) . DenseNet, on the other hand, lets the outputs of the first l layers be used as inputs to layer l + 1, as shown in equation (29) [x0,x1,⋯,xl−1] denoting the connection of the outputs of the first l layers: (27)xl=Hl(xl−1) (28)xl=Hl(xl−1)+xl−1 (29)xl=Hl([x0,x1,⋯,xl−1])

All the layers in front are densely connected to the layers behind, and this connection method is helpful to alleviate the problem of gradient vanishing, and it can effectively avoid overfitting during the network training process with good regularization effect. In addition, it can also be noted that DenseNet realizes feature reuse through the connection of features on channels, and this image feature linking method can omit many parameters and reduce the learning redundancy. These two features allow DenseNet to achieve better performance than classical networks such as ResNet with fewer total parameters and lower computational cost.

SENet is a network structure that focuses on channel relationships, different from the usual focus on the spatial dimension level to improve the network performance, its model has the ability to rely on the interrelationships between the channels to adaptively calibrate the features of the channels, and differentiate the importance of each feature channel in order to enhance the valuable feature information and suppress the irrelevant features, and its structure includes the two Squeeze and Excitation key operations, so it is named SENet [32].

F_tr for convolution, X and U denote the input feature channel and output feature channel, Fsq(⋅) denotes the squeezing function, Fex(⋅,W) denotes the excitation function, and a typical SENet module is represented in Fig.

In the squeezing module, the squeezing module is a global average pooling operation that turns spatial information into channel descriptors, as in Eq. (30), and the cth element in Z can be reduced by Fsq(⋅) shrinking the dimension H*W of U in the space, e.g., from 6 × 6 to 1 × 1: (30)Zc=Fsq(Uc)=1H×W∑i=1H∑j=1WUc(i,j)

The excitation module connects the pooled C-dimensional feature vector with two fully connected layers in sequence, the role is to downscale the feature to C/r and re-upscale it back to C respectively, and finally activate it with the Sigmoid function, and the output value is limited to be between 0 and 1, and then finally multiply the weight matrix and U to give the weights as follows (31), δ represents the ReLU function, and the output function is the dimension of the feature of W₁z through the W₂δ multiplied to get the result of the excitation module: (31)S=Fex(z,W)=σ(g(z,W))=σ(W2δ(W1z))

After the two modules are operated, then the C-dimensional features are fused with input U for the feature channel. In Eq. (32), u_c is a two-dimensional matrix, s_c is the computed weights, and the F_scale function is used to operate on these two values: (32)xc=Fscale(uc,sc)=uc×sc

As known from the previous section, the SENet module is simple-minded and very flexible, and it can be loaded into the existing network model architecture relatively easily, and the residual network models can all integrate the SE block into it as a nonlinear constant branch, so in this paper, we embed the SENet into DenseNet to improve the accuracy of pottery gesture recognition.

The network model in this thesis utilizes Leap Motion acquisition to detect the beginning keyframes of the depth image data, so as to obtain the whole segment of gesture motion data based on the number of beginning frames. The dense jump convolutional network structure is improved based on DenseNet, and a feature channel weight training module (SENet module) is added between the DenseNet dense block and the transition layer, which helps the network to focus on the effective feature parts and reduces the activation of irrelevant parts, in order to improve the accuracy of the gesture recognition during the pottery process.

The SD method, Semantic Difference Method, is a psychometric method in which the psychological feelings of the subjects are measured through language, and quantitative data on the subjects’ feelings can be obtained.

There were 15 females and 15 males among the test subjects, and the age range was mainly from 18 to 60 years old, 30 questionnaires were distributed, of which 30 were recovered, with an effective recovery rate of 100%. Through the statistics of the collected questionnaire data, the SD score table of 5 samples was obtained, as shown in Table 1. The SD score in the table is the average of the scores of each evaluation factor in all valid questionnaires, and the larger the value means that the scores are closer to the advantageous adjectives on the right side, and the smaller the value means that the scores are closer to the disadvantageous rating words on the left side.

By observing the mean value of the SD scores, the scores of the 13 evaluation factors are concentrated in the range of -0.001 to 0.2802, which indicates that the evaluators are satisfied with the overall quality of the experience of the five samples. Among them, in the evaluation of virtual pottery scene design factors, the respondents’ evaluation of the color factor is higher, and the mean score is higher than the rest of the evaluation factors, indicating that all the respondents are more satisfied with the overall evaluation of the perception of the color design of the samples, in the evaluation of the interaction design factors, the lowest score of the operation feedback, and all the respondents feedback that the operation feedback of the samples is too homogeneous, and in the evaluation of the exhibition design factors, the richness of the ceramic exhibits’ information In the evaluation of exhibition design elements, the ceramic exhibit information richness score of -0.001 is the lowest overall, indicating that the respondents are the least satisfied with the sample’s ceramic exhibit information richness.

Plotting the SD score curve based on the data from the SD score table is shown in Figure 4, which allows for a visual comparison of the differences in the scores of the SD scores of the five samples on the 13 evaluation factors. The fold line overall to the right (highest score) is ceramic sample 3, with a mean score of 0.2812.

Figure 4.

SD score curve

Sample 3 of the functional architecture design at a glance, the system is clear, the interface layout of the appropriate sparse and rhythm coexist. Sample 3’s color and exhibit display effect of the evaluation factor score is significantly higher than the other samples, sample 3 maintains the traditional ceramics in the unique traditional colors, unique color style reflects the traditional ceramic characteristics. In the design of the whole traditional pottery display scene fully embodies the principle of situational experience, visual and interactive show exquisite and efficient characteristics, the use of audio, video, description of the information and real-life reproduction of the combination of exhibition methods to present a digital exhibition of the real display effect, so that the audience to get an immersive experience at the same time to reduce the long time use of digital equipment burnout, to improve the audience’s sense of pleasure to use the product and the curiosity. However, Sample 3’s excessive exhibition content and high-functionality architecture makes it more costly for viewers to use. The overall lower curve (lowest score) among all the samples is Sample 1 Traditional Ceramics Virtual Showroom, with an average score of 0.02654. Sample 1 exhibits a single content, only a type of museum real-life reproduction, the entire digital exhibition is low interactivity, low interest, lack of attraction, relative lack of functionality, in the subsequent scene design should be to enhance the interactivity between the pottery and the audience.

Fig. 5 is the satisfaction of users of traditional ceramic scene space, and the questionnaire results of the satisfaction of users of traditional ceramic scene are statistical, in which the index evaluation results are mostly positive, and the indicators in the figure are D1 (scene scale suitability), D2 (acceptability), D3 (scene space utilization), D4 (scene space enclosure), D5 (aesthetic value of traditional ceramic technology), D6 (scene construction perfection), D7 (pottery tool perfection), D8 (continuity), D9 (traditional cultural activity), D10 (traditional cultural environment dependence), D11 (activity frequency), D12 (cultural atmosphere), D13 (social influence), D14 (operation management), D15 (mechanism soundness), D16 (willingness to participate in activities), D17 (participation in activities), D18 (support), D19 (cultural identity). The average value of the 19 user satisfaction was 23.6316, and the scores of each index ranged from 13~35.

Figure 5.

Customer satisfaction of traditional ceramic scene space

By evaluating the various sub-element level indicators in the previous section, the final scores are calculated and plotted together with the SD mean semantic distribution, and the final results are shown in Figure 6. 1)

Positive factor analysis

The overall positive factor of the traditional ceramic art virtual scene space accounts for the majority of the overall positive factor, and the absolute value of the overall high, in which the frequency of activities to reach the maximum degree of satisfaction state, the factor evaluation of 2. The traditional ceramic art virtual scene designed in this paper makes the audience’s frequency of activities to improve, and has been recognized by the audience. But compared to the above indicators, the traditional ceramic virtual scene in the scene scale appropriateness, traditional culture alive, operation and management, the degree of soundness of the mechanism and the degree of cultural identity appears to be slightly inferior, the factor evaluation of 0.245, 0.496, 0.615, 0.079 and 0.344. The reason for this is that the degree of soundness of the mechanism score and the audience’s degree of cultural identity has a very strong relationship between the audience, and many of the audience see less ceramic cultural value parts and do not have a strong sense of identity with the cultural environment they are in.

Figure 6.

The average semantic distribution of traditional ceramic art scene space

In this section, IPA analysis will be used to combine the data corresponding to the SD data in the previous section, and the quantitative data corresponding to the SD data and the weight data will be transformed into a qualitative graphical language, so as to facilitate a more accurate classification of the 32 evaluation factors, and to provide a basis for the strategy proposal and optimization design of the later section. Summarize the traditional ceramic scene evaluation and user satisfaction indicators, the traditional ceramic scene evaluation using A1-A13 representation.

This paper uses the constructed IPA model to categorize and analyze the traditional ceramic scene evaluation and user satisfaction indicators respectively. The reference value of the horizontal coordinate importance is 0.031, and the reference value of the vertical coordinate satisfaction is different, which is the average value of the user satisfaction score of each sample.

The reference value of traditional ceramic art scene space vertical coordinate satisfaction is SD score mean 0.4126. Figure 7 shows the traditional ceramic scene user satisfaction, and the distribution of its 19 sub-factor indicators is shown below. There are 3 indicators in Zone Ⅰ, which are D3 (Scene Space Usage Rate), D9 (Traditional Culture Alive), D10 (Traditional Culture Environment Dependence), these 3 indicators are the ones that are considered important by the experts and relatively satisfied by the users, and attention should be paid to the maintenance of the advantages of these indicators when proposing optimization strategies. There are 7 indicators in Zone Ⅱ, which are the indicators that experts consider relatively unimportant, but user satisfaction is still high, and these indicators are the advantages that users of traditional ceramic scenes are satisfied with, so it is enough to continue to maintain them. Ⅲ area of the five indicators for the importance of the lower, user satisfaction is also relatively low indicators, is secondary to consider the optimization and enhancement of the indicators. Ⅳ area fell into a total of 4 indicators, these 4 indicators are experts believe that important, but the user satisfaction is lower indicators, is the optimization strategy for the primary target.

Figure 7.

Traditional ceramic scene user satisfaction

Figure 8 shows the evaluation of the traditional ceramic scene, categorizing its 13 indicators. Ⅰ and Ⅱ area of a total of 4 indicators, these indicators for the advantages of traditional ceramic art scene space, continue to maintain can. Ⅲ area of the 2 indicators for the important lower, traditional ceramic art scene space is also relatively low indicators, is secondary consideration for optimization and enhancement of the indicators. Ⅳ area falls into a total of 3 indicators, respectively, A5 (ease of use), A7 (information display level), A10 (system functionality), these 3 are the optimization strategy for the primary target.

Figure 8.

Traditional ceramic scene evaluation