Method and non-transitory, computer-readable storage medium for deep learning model based product matching using multi modal data

Abstract

Methods and systems for generating a plurality of matching items that match a reference item are disclosed. The method includes first determining reference attribute data for the reference item, where the reference attribute data is multimodal. Next, selecting a deep learning multimodal matching model from a plurality of candidate multimodal matching models. The selected deep learning multimodal matching model has a first deep learning neural network (DLNN) for processing data having a first data mode and a second DLNN analyzer for processing data having a second data mode. Then, matching a potential matching item to the reference item using the selected deep learning multimodal matching model to generate a match score, where the match score is computed based on the reference attribute data for the reference item and attribute data for the potential matching item. Finally, adding the potential matching item to the plurality of matching items based on the match score.