Intelligent grading method and system for pulmonary nodules based on multi-modal feature fusion

Abstract

Provided is an intelligent grading method and system for pulmonary nodules based on multi-modal feature fusion, including: obtaining ROI and VOI of pulmonary nodules based on chest CT examination images and examination reports by utilizing clinical multi-modal data from physical examination population, designing a multi-task feature extraction network based on attention mechanism, to obtain radiomics features and deep image features from the ROI and VOI; designing a cross-modal feature fusion method based on graph representation learning, designing a multi-modal information extraction method, obtaining specific feature representations and graph structures of modalities, and then fusing the feature representations and the graph structures; and proposing an optimization and clinical verification method of pulmonary nodule grading GCN model based on self-supervised learning, to realize fine grading of pulmonary nodule malignancy with slight differences, thereby providing a new approach to design of fine-grained classification algorithms.