Machine-learning models for extracting and classifying image content, and augmenting image based on same

Abstract

An online system that displays items from an item catalog to users supplements content displayed for one or more of the items with information extracted from images of the items. For a particular item in the item catalog, the online system performs image processing, such as optical character recognition, on one or more images of the item to extract text phrases from the images. For each extracted text phrase, the system then uses a trained model to score the text phrase as being a viable informational message. If the score for a text phrase is above a threshold, the online system augments content displayed in a user interface for the item with the text phrase. The online system may decide whether to supplement content for the item with an extracted text phrase based on the output of a predictive model.