Method for feature detection of complex defects based on multimodal data

Abstract

The present disclosure disclose a method for feature detection of complex defects based on multimodal data, including feature extraction of multimodal data, multimodal feature cross-guided learning, multimodal feature fusion, and defect classification and regression. Feature extraction networks for multimodal two-dimensional data are constructed first, and a defect data set is sent to the networks for training; during training, cross-guided learning is implemented by using a multimodal feature cross-guidance network; then feature fusion is performed by using a weight adaptive method; and finally a defect detection task is implemented by using a classification subnetwork and a regression subnetwork. In the present disclosure, fusion of the multimodal data in a process of feature detection of the complex defects can be implemented efficiently, a capability of detecting the complex defects in an industrial environment can be improved more effectively, and production efficiency in an industrial manufacturing process is ensured.