Amazon Technologies, Inc.

Mingyue Shang

Bing Xiang

Atul Deo

Andrew Oliver Arnold

Yuchen Tian

Ankur Deepak Desai

Zijian Wang

Praphruetpong Athiwaratkun

Sudipta Sengupta

Ramesh M Nallapati

Parminder Bhatia

Matthew Lee

Srinivas Iragavarapu

Yanitsa Donchev

Vamshidhar Krishnamurthy Dantu

Ming Tan

Random token segmentation may be implemented for next token prediction. Text data may be received for training a machine learning model to predict a next token given input text tokens. Multiple tokens may be determined from the text data. Different ones of the multiple token may be randomly segmented in to sub-tokens. The machine learning model may then be trained using the multiple tokens including the respective sub-tokens as a training data set.

Random token segmentation for training next token prediction models

Pre-fix matching may constrain the generation of next token predictions. Input text to perform a next token prediction may be received. Multiple tokens may be determined from the input text, including a partial token. From possible tokens, one or more matching possible tokens with the partial token may be identified. Next token predictions may then be filtered using the identified possible tokens in order to ensure that the partial token is matched.

Constrained prefix matching for generating next token predictions

Growing community of inventors

Jersey City, NJ, United States of America

Mingyue Shang