Multimodal web page content classification with knowledge graph-based explainability

Abstract

A multimodal web page content classification system ('classification system') uses summarization, extraction, and knowledge graph generation to classify web page content. For each of multiple modalities, the classification system summarizes web page content corresponding to that modality and then identifies key phrase/concept pairs in that modality summary. The classification system builds a knowledge graph from the key phrase/concept pairs as well as known concept/category pairs. Finally, the classification system invokes a classifier on a representation of the knowledge graph to obtain a category classification, resulting in an interpretable classification via the modality summaries and knowledge graph.