Microsoft Technology Licensing, LLC

Shao Kun Deng

Neelakantan Sundaresan

Alexey Svyatkovskiy

Michele Tufano

Shengyu Fu

Xiaoyu Liu

Colin Bruce Clement

Shuvendu K Lahiri

Xin Shi

Matthew Glenn Jin

Mark Alistair Wilson-Thomas

David Ellis Pugh

Miltiadis Allamanis

German David Obando Chacon

Vivian Julia Lim

Daya Guo

Anqi Li

Jonathan Keith Simmons

Shwetha Srinath

Jin Woo Jang

An automated program repair tool utilizes a neural transformer model with attention to predict the contents of a bug repair in the context of source code having a bug of an identified bug type. The neural transformer model is trained on a large unsupervised corpus of source code using a span-masking denoising optimization objective, and fine-tuned on a large supervised dataset of triplets containing a bug-type annotation, software bug, and repair. The bug-type annotation is derived from an interprocedural static code analyzer. A bug type edit centroid is computed for each bug type and used in the inference decoding phase to generate the bug repair.

Automated program repair tool

A deep learning model trained to learn to predict source code is tuned for a target source code generation task through reinforcement learning using a reward score that considers the quality of the source code predicted during the tuning process. The reward score is adjusted to consider code-quality factors and source code metrics. The code-quality factors account for the predicted source code having syntactic correctness, successful compilation, successful execution, successful invocation, readability, functional correctness, and coverage. The source code metrics generate a score based on how close the predicted source code is to a ground truth code.

Code generation through reinforcement learning using code-quality rewards

A code completion tool uses a neural transformer model to generate candidate sequences to complete a line of source code. The neural transformer model is trained using a conditional language modeling objective on a large unsupervised dataset that includes source code programs written in several different programming languages. The neural transformer model is used within a beam search that predicts the most likely candidate sequences for a code snippet under development.

Multi-lingual line-of-code completion system

A constrained decoding technique incorporates token constraints into a beam search at each time step of a decoding process in order to generate viable candidate sequences that are syntactically and semantically correct. The token constraints identify source code tokens or sequences of tokens that should appear in a candidate sequence. The token constraints are generated from checking whether a token predicted at each decoding step is feasible for a partial solution based on the production rules of the grammar of the programming language, the syntactic correctness of a partial sequence, and/or static type correctness.

Constrained decoding for source code generation

A code completion tool uses a neural transformer model with attention to generate syntactically-correct candidates with holes to complete a partially-formed code snippet. The model is trained to predict the expansion of non-terminal symbols of the production rules of the underlying grammar of the code snippet without being constrained to a left-to-right expansion order. A hole is a non-terminal symbol of the grammar of a programming language that marks a position in a candidate where the code completion engine is not certain of the production rule that should be used to expand the non-terminal symbol. The hole allows the code completion engine to expand other non-terminal symbols in a candidate and allow the user to guide the expansion of the holes in a candidate.

Code completion with holes

A test-driven development system utilizes a neural transformer model with attention to generate method bodies for a focal method given its associated test cases, and optionally a method signature and a docstring of the focal method. The candidate method bodies are validated for syntactic correctness, tested using the given test cases, and tested with a donor class in a target system. Those candidate method bodies passing the validation and testing are then ranked based on a PLUM score that analyzes the candidate method bodies against various quality and performance metrics.

Automating test-driven development with transformers

Edit automation enhancements may be implemented in source code editors and other text editors. Provisional selections that indicate user intentions are submitted to a suggestion generator with other edit context information, to improve the quality of generated text suggestions and reduce the cognitive load on users. A provisional selection may include a highlighted completion list entry, or document text targeted by a hovering cursor, or metainformation text targeted by the hovering cursor, for example. An inline grey text suggestion driven by provisional selection may be displayed simultaneously with completion list suggestions that were created without regard to provisional selection. Suggestions driven by provisional selection may be interleaved with existing document text. Suggestions may be accepted fully in one gesture, or in parts. Suggestions may be edited by a user before being accepted, driving further suggestion refinement. Multiple suggestions may be displayed simultaneously, reducing pressure on the suggestion generator.

Provisional selection drives edit suggestion generation

Language interoperability between source code programs not compatible with an interprocedural static code analyzer is achieved through language-independent representations of the programs. The source code programs are transformed into respective intermediate language instructions from which a language-independent control flow graph and a language-independent type environment is created. A program compatible with the interprocedural static code analyzer is generated from the language-independent control flow graph and the language-independent type environment in order to utilize the interprocedural static code analyzer to detect memory safety faults.

Growing community of inventors

Bellevue, WA, United States of America

Shao Kun Deng