Determining indications of visual abnormalities in an unstructured data stream

Abstract

A corpus of textual data records, labeled by experts as corresponding to a defined characteristic, that comprise descriptions of problems with an item are collected. A language model generates a plurality of n-grams from the corpus. Frequently occurring n-grams are analyzed using a zero-shot learning model to determine similarity to the defined characteristic. N-grams highly similar to the defined characteristic may be selected as defined phrases. N-grams highly similar to another characteristic may also be selected to reduce false positives. The zero-shot model may also be used to determine a weighting factor for each defined phrase for each record. A relevance score is determined for a record by multiplying the weighting factors for each phrase that has a similarity score relative to the record above a threshold based on the expert labeling. The relevancy score may be used to automatically diagnose problems with the item.