System and method for text moderation via pretrained transformers

Abstract

The present teaching relates to method, system, medium, and implementations for text processing. Upon receiving input data including a plurality of text strings, a plurality of manipulated text strings are generated for each of the plurality of training text strings by first applying a manipulation to each of at least one original token in the text string to generate a manipulated token, where the original token has a ground truth token label and then determining, with respect to each manipulated token, a ground truth action which, when applied to the manipulated token, yields the original token with the ground truth token label. Training data are generated with a plurality of training data packs, each of which corresponds to one of the plurality of text strings in the input data and includes a manipulated text string with at least one manipulated token, at least one ground truth token label, and at least one ground truth action which, when applied to the at least one manipulated token produces the at least one ground truth token label. The training data are for training text moderation models that facilitate text moderation.