Microsoft Technology Licensing, LLC

Colin Bruce Clement

Neelakantan Sundaresan

Alexey Svyatkovskiy

Dawn Drain

Michele Tufano

Roshanak Zilouchian Moghaddam

Shuai Lu

Guillermo Serrato Castilla

Shengyu Fu

Shao Kun Deng

Xiaoyu Liu

Duyu Tang

Andrei Zlotchevski

Yevhen Mohylevskyy

Xin Shi

Chen Wu

Matthew Glenn Jin

Aaron Yue-chiu Chan

Anant Girish Kharkar

Maoquan Wang

Shubham Chandel

Yiding Tian

Dongjiang You

David Alberto Nader Palacio

Yufan Huang

A code coverage prediction system utilizes a neural transformer model with attention to generate a sequence of coverage symbols given a focal method and a test case. The sequence of coverage symbols indicates whether a line of source code is covered by the test case, missed by the test case or unreachable. The sequence of coverage symbols is aligned with the focal method to produce a coverage-annotated focal method that associates a predicted coverage symbol with each line of source code in the focal method.

Predicting code coverage without execution

An example generator tool generates an example illustrating correct usage of a command of a command line interface. A command may include a command name, zero or more subcommands, and one or more parameters with a corresponding parameter value. A template containing the correct syntax of the command is obtained from a template database. Parameter values for the template are generated from a neural transformer with attention given the command template.

Neural command line interface example generation

A code completion tool uses a neural transformer model with attention to generate candidate sequences to complete a method body of a method signature. The neural transformer model is trained with source code programs and natural language text. The neural transformer model learns the meaning of a method name, its corresponding method parameters and types from a large corpus of unsupervised dataset of source code methods and a supervised dataset of tasks including source code constructs in combination with natural language docstrings to infer a candidate sequence of subtokens that represent a method body for a particular method signature.

Neural method completion based on natural language and source code

A false positive vulnerability system detects whether a software vulnerability identified by a static code vulnerability analyzer is a true vulnerability or a false positive. The system utilizes deep learning models to predict whether an identified vulnerability is accurate given the source code context of the identified vulnerability. A neural encoder transformer model is trained to classify a false positive given the method body including the identified vulnerability. A neural decoder transformer model is trained to predict a candidate line-of-code to complete a prompt inserted into the context of the identified vulnerability. The candidate line-of-code that successfully completes the prompt is used as a signal to identify that the identified vulnerability is a false positive.

False positive vulnerability detection using neural transformers

A technique to prevent a prompt injection attack utilizes a security agent to sign a large language model prompt with a secret that is isolated from the user application or device that generates a user prompt. The secret is tailored for a specific user identifier and session identifier. The large language model is instructed to repeat the secret in each response. The security agent retrieves the response from the large language model and checks for the secret. When the secret is not part of the response, an error message is forwarded to the user application instead of the response.

Signing large language model prompts to prevent unintended response

A neural transcompilation model is tested with a set of syntax unit tests to determine the syntax elements of a source code program written in a source programming language that fail to translate properly into a target programming language. The syntax elements having a translation defect are identified and ranked according to a translation failure rate. The neural transcompilation model is then fine-tuned with training samples of the syntax elements having the highest translation failure rates and their paired correct translation in order to teach the model to learn the association between the syntax elements of a source programming language causing translation defects and its correct translation in a target programming language.

Syntax unit testing and fine-tuning of neural transcompilation models

A user query for information regarding data of a codebase is answered by a large language model given a prompt that includes examples of code segments from the codebase that are similar to the user query. The code segments from the codebase are associated with metadata that includes both natural language text and source code. The search for the examples of code segments from the codebase is based on embeddings of code segments and associated metadata that are closely similar to an embedding of the user query and context.

Code search for examples to augment model prompt

A transfer learning system is used for the development of neural transformer models pertaining to software engineering tasks. The transfer learning system trains source code domain neural transformer models with attention in various configurations on a large corpus of unsupervised training dataset of source code programs and/or source code-related natural language text. A web service provides the trained models for use in developing a model that may be fine-tuned on a supervised training dataset associated with a software engineering task thereby generating a tool to perform the software engineering task.

Transfer learning system for automated software engineering tasks

Custom source code generation models are generated by tuning a pre-trained deep learning model by freezing the model parameters and optimizing a prefix. The tuning process is distributed across a user space and a model space where the embedding and output layers are performed in the user space and the execution of the model is performed in a model space that is isolated from the user space. The tuning process updates the embeddings of the prefix across the separate execution spaces in a manner that preserves the privacy of the data used in the tuning process.

Custom models for source code generation via prefix-tuning

A neural classifier model is used to detect cybersecurity vulnerabilities in the source code predicted by a deep learning code generation model having been trained on source code possibly containing security bugs. Upon the classifier model classifying a given source code snippet as likely containing a cybersecurity vulnerability, a proposed repair for the cybersecurity vulnerability is predicted from a neural decoder transformer model having been trained on non-vulnerable source code. The neural decoder transformer model is used to predict source code that repairs the cybersecurity vulnerability given the source code classified with a cybersecurity vulnerability.

Source code vulnerability detection and repair through machine learning

A debugging tool identifies the smallest subset of an input sequence or rationales that influenced a neural language model to generate an output sequence. The debugging tool uses the rationales to understand why the model made its predictions and in particular, the particular input tokens that had the most impact on the output sequence. In the case of erroneous output, the rationales are used to alter the input sequence to avoid the error or to tailor a new training dataset to retrain the model to improve its performance.

Debugging tool for code generation neural language models

A constrained decoding technique incorporates token constraints into a beam search at each time step of a decoding process in order to generate viable candidate sequences that are syntactically and semantically correct. The token constraints identify source code tokens or sequences of tokens that should appear in a candidate sequence. The token constraints are generated from checking whether a token predicted at each decoding step is feasible for a partial solution based on the production rules of the grammar of the programming language, the syntactic correctness of a partial sequence, and/or static type correctness.

Constrained decoding for source code generation

An automated system for translating source code written in one programming language into a different programming language utilizes a neural transformer with attention trained on semi-supervised data. The model is jointly pre-trained with a masked language model objective and an autoregressive objective on a large unsupervised source code corpus to learn to comprehend the syntactic structure and semantics of source code. The pre-trained model is then fine-tuned with a token-type prediction objective and an autoregressive objective on supervised translation tasks and data augmented tasks to learn to translate source code from one programming language into a different programming language.

Semi-supervised translation of source code programs using neural transformers

A neural transformer model with attention is trained to predict candidates to complete a line of source code with a zero-inference capability. The model is trained on an unsupervised training dataset that includes features from source code written in multiple programming languages. The features include a file-level context and a local context, where the file-level context includes a global context, a class context, a function context, and/or a method context for each class, function and/or method of the source code programs used in the training dataset. The local context includes method bodies, function bodies, and/or stand-alone code of main method routines. From these features, the model is able to learn to predict an ordered sequence of code elements that complete a line of source code in a programming language seen and not seen during training.

Growing community of inventors

Seattle, WA, United States of America

Colin Bruce Clement