Oracle International Corporation

Omid Mohamad Nezami

Tuyen Quang Pham

Thanh Tien Vu

Budhaditya Saha

Mark Edward Johnson

Thanh Long Duong

Cong Duy Vu Hoang

Poorya Zaremoodi

Ying Xu

Aashna Devang Kanuga

Shivashankar Subramanian

Shubham Pawankumar Shah

Techniques are provided for augmenting training data using gazetteers and perturbations to facilitate training named entity recognition models. The training data can be augmented by generating additional utterances from original utterances in the training data and combining the generated additional utterances with the original utterances to form the augmented training data. The additional utterances can be generated by replacing the named entities in the original utterances with different named entities and/or perturbed versions of the named entities in the original utterances selected from a gazetteer. Gazetteers of named entities can be generated from the training data and expanded by searching a knowledge base and/or perturbing the named entities therein. The named entity recognition model can be trained using the augmented training data.

Training data augmentation using gazetteers and perturbations to facilitate training named entity recognition models

In some aspects, a computing device may receive, at a data processing system, a set of utterances for training or inferencing with a named entity recognizer to assign a label to each token piece from the set of utterances. The computing device may determine a length of each utterance in the set and when the length of the utterance exceeds a pre-determined threshold of token pieces: dividing the utterance into a plurality of overlapping chunks of token pieces; assigning a label together with a confidence score for each token piece in a chunk; determining a final label and an associated confidence score for each chunk of token pieces by merging two confidence scores; determining a final annotated label for the utterance based at least on the merging the two confidence scores; and storing the final annotated label in a memory.

System and techniques for handling long text for pre-trained language models

Growing community of inventors

Sydney, Australia

Omid Mohamad Nezami