Amazon Technologies, Inc.

Yuchen Tian

Sudipta Sengupta

Rama Krishna Sandeep Pokkunuri

Bing Xiang

Ramesh M Nallapati

Parminder Bhatia

Matthew Lee

Srinivas Iragavarapu

Zijian Wang

Yanitsa Donchev

Praphruetpong Athiwaratkun

Atul Deo

Andrew Oliver Arnold

Ankur Deepak Desai

Vamshidhar Krishnamurthy Dantu

Sri Ranga Akhilesh Bontala

Yaser Al-Onaizan

Syed Kashif Hussain Shah

Kalpit Dixit

Jie Ma

Sujan Kumar Gonugondla

Jiacheng Guo

Hantian Ding

Ramana Keerthi

Zixuan Lin

Mingyue Shang

Pramod Chandra Samudrala

Himani Amrish Shah

Haifeng Qian

Xiaokai Wei

Bohan Yao

Techniques for using a quantized and/or fused model are described. In some examples, a service is to receive a request to use a trained model, the request including input data; apply a trained model to the input data, the application of the trained model includes fusing one or more matrix multiplication operations with element-wise operations; and output a result from the trained model.

Large language models utilizing element-wise operation-fusion

A system for providing code suggestions according to licensing criteria is described. The system comprises computing devices that implement a code suggestion service. The code suggestion service receives a request that specifies licensing criteria via an interface of the code suggestion service. The code suggestion service determines respective licenses for respective source code files according to a source code attribution database from parsing the plurality of source code files that are applicable to the plurality of source code files. The code suggestion service generates a set of candidate code suggestions based, at least in part, on the plurality of source code files. The code suggestion service determines code suggestions from the set of candidate code suggestions that satisfy the licensing criteria based on the respective licenses. The code suggestion service provides the code suggestions determined from the set of candidate source code files that satisfy the licensing criteria.

Software license-based code suggestions

Documents may be evaluated to determine inclusion of one or more types of information, such as personal information. Within the documents containing the one or more types of information, individual spans of information may be replaced with alternative data, which may maintain a type of content associated with the spans. Datasets of documents may be created where one or more types of information have been removed. These datasets may also be evaluated to determine whether one or more elements are over or under represented and then the datasets may be modified, such as by adding or removing documents, to provide an improved balance or distribution of the one or more elements.

Privacy preservation within datasets

Evaluation data sets may be programmatically generated for code generation models. An evaluation data set is obtained that includes items that correspond to different evaluation tests for a code generation system. The individual items of the evaluation data set maybe converted, including the conversion of a function signature for the items, the test statements for the items and using a code generation system to generate the body of the function.

Programmatically generating evaluation data sets for code generation models

Pre-fix matching may constrain the generation of next token predictions. Input text to perform a next token prediction may be received. Multiple tokens may be determined from the input text, including a partial token. From possible tokens, one or more matching possible tokens with the partial token may be identified. Next token predictions may then be filtered using the identified possible tokens in order to ensure that the partial token is matched.

Constrained prefix matching for generating next token predictions

Growing community of inventors

Santa Clara, CA, United States of America

Yuchen Tian