Research on robotic mechanical power sensing model based on multimodal sensor fusion

There is an increasing demand for multimodal sensor fusion in the field of robot fine manipulation, but how to design efficient and convenient perception prediction modules is still a challenging problem. The model designed in this paper focuses on fully exploiting the complementarity and common distribution of audio-visual and tactile modal data, and combines with generative adversarial networks to realize multimodal collaborative generation of perception. As the dictionary size K increases, the recognition accuracies of the model in this paper are higher than the baseline model OSLSR at different K stages, and the generalization ability of the model under different parameters is verified. And regardless of sparsity, the recognition of this paper’s model is significantly better than JKSC and AMDL. When T=5, the maximum recognition result is 0.953, which is higher than the recognition performance of the remaining two models. When T>5, this paper’s model begins to show a decreasing trend, but still higher than the other algorithms. Combining the results of all the experiments, it can be concluded that the model in this paper better embodies the multimodal co-generative perception of the robot.