Microsoft Technology Licensing, LLC

Sumit Gulwani

Vu Minh Le

Ranvijay Kumar

Rishabh Singh

Gustavo Araujo Soares

Daniel Adam Perelman

Mark Alistair Wilson-Thomas

Abhishek Udupa

Arjun Radhakrishna

Oleksandr Polozov

Ashish Tiwari

Euan Peter Garden

Chairy Chiu Ying Cheung

Ramarathnam R Venkatesan

Benjamin Zorn

Prateek Jain

Daniel Galen Simmons

Mark Marron

Mohammad Raza

Dany Rouhana

William Robert Harris

Saswat Padhi

José Pablo Cambronero Sánchez

Ivan Radicek

Titus Barik

Butler W Lampson

Giovanni Moises Della-Libera

Nebojsa Jojic

Trishul A Chilimbi

Geoffrey G Zweig

David Ellis Pugh

Krishna Kumar Mehra

Adam Tauman Kalai

Vladimir Jojic

Piyush Arora

Omer Tamuz

Saurabh Srivastava

Kevin Michael Ellis

Wode Ni

Ameya Sanjay Ketkar

Gust Ben Anneloes Verbruggen

Aditya Krishna Menon

Eric Cohen

Yun-Cheng Ju

Kris Kuppuswamy Ganjam

Nikolai Tillmann

Chad B Rothschiller

Kaushik Chakrabarti

Daniel Morris

Daniel C Battagin

Madanlal S Musuvathi

Sriram Kottarakurichi Rajamani

Nachiappan Nagappan

Jonathan Paul De Halleux

Theresa Ann Estrada

Kevin David James Grealish

Prabhdeep Singh

Stan Birchfield

Saurabh Kumar Tiwary

Hao Dang

Ashley Nathan Feniello

Craig Allen Lee

Weide Zhong

Chiu Ying Cheung

Akash Lal

Peter Groenewegen

Carina Suzana Negreanu

Aditya Nori

Trishul Amit Madhukar Chilimbi

Joseph J LaViola, Jr

David P Walker

Xiang Gao

Shraddha Govind Barke

Zhouying Fan

Itai Rosenberger

Salman Shaukat Cheema

Rahul Srinivasan

Florian Franz Zuleger

Michael Vollmer

Sudeep Dilip Juvekar

Raluca Carmen Sauciuc

Anders Miltner

Sigmund Isy Cherem

Alan Thomas Leung

Ashwin Kalyan Vijayakumar

Yichen Wang

Dario Pavlovic

Abhishek Mohta

Jeffrey Weston

Dana Drachsler Cohen

Anna Fariha

Christopher Benedict

Rohan Jayesh Bavishi

Rodrigo Moreira-Silveira

Kiarash Rahmani

Dmitri O Danilov

Mikael Mayer

Some embodiments generate prompts and submit them in queries to a language model trained on code to perform automated program repair. Some embodiments fix syntactic mistakes and semantic mistakes by combining multimodal prompts, iterative querying, test-case-based selection of few-shots, and program chunking. In some cases, edit distance is minimized between an initial flawed program and the automatically created improved version of that program. The initial flawed program is obtained from a programming student, or from a source code generator.

Program improvement using large language models

Techniques are described herein that are capable of creating a language-agnostic computer program repair engine generator. A context-free grammar is annotated to identify token(s) that are likely to be included in or excluded from a computer program in a manner that violates the context-free grammar. A language-agnostic computer program repair engine generator is created that is configured to generate a parser. The repair engine generator is configured to create a repair engine that: converts the candidate string into repaired strings that neither violate the context-free grammar nor violate a criterion for a valid computer program; calculates differences between the candidate string and the respective repaired strings; and replaces the candidate string with a designated repaired string based at least in part on the difference between the designated repaired string and the candidate string being less than or equal to a difference threshold.

Language-agnostic computer program repair engine generator

Some embodiments perform fully unsupervised string error detection and repair, for strings containing both syntactic errors and semantic errors. In some embodiments, string error mitigation includes learning a regular-expression-based pattern that characterizes at least a predefined fraction of a set of string data values, identifying and masking a semantic substring, deriving edits to repair a string which contains the semantic substring, and replacing a mask value with a replacement substring that conforms to the pattern. Some embodiments utilize a language model. Some embodiments include or utilize a pattern learner, one or more learned significant patterns, and a repair engine. Some embodiments exploit program execution information to generate patterns when significant patterns are not initially found.

String data error detection and repair

Edit automation functionality generalizes edits performed by a user in a document, locates similar text, and recommends or applies transforms while staying within a current workflow. Source code edits such as refactoring are automated. The functionality uses or provides anchor target lists, temporal edit patterns, edit graphs, automatable edit sequence libraries, and other data structures and computational techniques for identifying locations appropriate for particular edits, for getting transforms, for selecting optimal transforms, for leveraging transforms in an editing session or later, and for displaying transform recommendations and results. The edit automation functionality enhances automation subtool generation, discoverability, and flexibility, for refactoring, snippet insertion, quick actions in an integrated development environment, and other automatable edit sequences.

Edit automation using an anchor target list

Embodiments use a multi-modal approach to generate software programs that match a solution program description. The solution program description may include natural language, input-output examples, partial source code, desired operators, or other hints. Some embodiments use optimized prompts to a pre-trained language model to obtain initial candidate programs. Maximal program components are extracted and then recombined variously using component-based synthesis. Beam search reduces a solution program search space by discarding some candidates from a given synthesis iteration. Relevance metrics, string similarity metrics, operator frequency distributions, token rareness scores, and other optimizations may be employed. By virtue of optimizations and the multi-modal approach, a solution program may be obtained after fewer iterations than by use of a language model alone. The multi-modal approach is domain agnostic, as illustrated by examples using regular expression and cascading style sheet selector domain specific languages.

Multi-modal program inference

Various technologies described herein pertain to automated data splitting using predictive program synthesis. Input-only examples for splitting an input column of an input data set can be received. The input-only examples can include example entries from the input column of the input data set to be split into multiple output columns without specification of how the example entries are to be split into multiple output columns. Further, a program for splitting the input column of the input data set into the multiple output columns can be synthesized based on the input-only examples. The program can be synthesized, given the input-only examples, in a domain-specific language (DSL) for splitting an entry into a tuple of fields utilizing a predictive program synthesis algorithm. Moreover, the program can be executed on the input data set to split the input column of the input data set into the multiple output columns.

Automatic splitting of a column into multiple columns

Edit automation using a temporal edit pattern

A method for transforming strings includes identifying one or more candidate example input strings from a database including a set of input strings. The candidate example input strings are presented for example transformation. For one or more of the candidate example input strings, an example output string corresponding to that example input string is received, where each example input string and its corresponding example output string define a transformation example in an example set. A string transformation program is generated based on transformation examples in the example set.

Example management for string transformation

Systems, methods, and computer-executable instructions for synthesizing programs using a prefix of an output. A specification of a task to synthesize a program in a domain specific language (DSL) is received. The specification includes an input and a corresponding prefix of the output. Programs for the task are synthesized. The synthesizing includes generating sub-goals based on the specification. Each of the synthesized programs include a solved subset of sub-goals and each sub-goal includes a symbol in the DSL. The symbol is transformed based on the DSL. The sub-goals are solved based on the transforming of the symbol using the input and the corresponding prefix of the output to generate the synthesized programs. The prefix of the output matches a prefix of an output from each of the plurality of synthesized programs.

Output prefix specification transformers

A synthesis procedure learns program transformations for a text document, on-the-fly during an edit session, from examples of concrete edits made during the edit session and from an unsupervised set of additional inputs. The additional inputs are derived from explicit feedback from the user and inferred feedback from the user's behavior during the edit session. A reward score, based on anti-unification and provenance analysis, is used to classify the additional inputs as either a positive input or a negative input. Outputs are generated for the positive inputs that are consistent with the existing examples and then used to synthesize a new program transformation. The program transformations are then used to generate code edit suggestions during the edit session.

Feedback-driven semi-supervised synthesis of program transformations

Described herein are various technologies pertaining to identifying boundaries of a substring in a log file, wherein the substring is to be extracted from the log file and used to construct a table. An indication is received that a user has selected a beginning boundary of the substring, and the ending boundary of the substring is automatically identified.

Identifying boundaries of substrings to be extracted from log files

Various technologies described herein pertain to interactive data splitting. A program for splitting an input column of an input data set into multiple output columns can be synthesized based on input-only examples. The program can further be generated based on various user input; thus, the user input can guide the synthesis of the program. Moreover, the program can be executed on the input data set to split the input column of the input data set into the multiple output columns.

Interactive splitting of a column into multiple columns

Flexible yet efficient 'find' operations search source code for matches to a general pattern after a developer provides an example string that matches the pattern, without requiring the developer to write a regular expression or script that will implement the desired pattern. Example-driven find-replace functionality uses regular expressions or other pattern match codes, and scripts or other transforms, which are synthesized automatically from examples provided by a developer. This technology allows the developer to focus on workflow inside an integrated development environment instead of breaking focus to search for external documentation, or unfortunately foregoing the flexibility and power of regular expressions and scripts. Synthesizer outputs may be directly or indirectly ranked through user feedback, allowing their refinement. Find match generality may be controlled, e.g., by specifying regex star positions or star counts. Entry of guiding examples may be assisted by autocompletion. Performance criteria are also described.

Source code text replacement by example

Examples are disclosed herein that relate to string transformation sub-program suggestion. One example provides, at a computing device, a method comprising receiving a dataset having a plurality of input strings; receiving a declarative user input associated with a desired string transformation sub-program to be performed using the dataset as input; in a search space having a plurality of string transformation sub-programs, reducing the search space based on (i) the user input and (ii) one or more of the plurality of input strings of the dataset, to thereby form a reduced search space, the reduced search space having at least one string transformation sub-program that is configured to transform one or more of the plurality of input strings of the dataset; and selecting, from the reduced search space, one or more suggested string transformation sub-programs.

String transformation sub-program suggestion

A computing device includes a storage machine holding instructions executable by a logic machine to generate multi-string clusters, each containing alphanumeric strings of a dataset. Further multi-string clusters are generated via iterative performance of a combination operation in which a hierarchically-superior cluster is generated from a set of multi-string clusters. The combination operation includes, for candidate pairs of multi-string clusters, generating syntactic profiles describing an alphanumeric string from each multi-string cluster of the candidate pair. For each of the candidate pairs, a cost factor is determined for at least one of its syntactic profiles. Based on the cost factors determined for the syntactic profiles, one of the candidate pairs is selected. The multi-string clusters from the selected candidate pair are combined to generate the hierarchically-superior cluster including all of the alphanumeric strings from the selected candidate pair of multi-string clusters.

Syntactic profiling of alphanumeric strings

Growing community of inventors

Redmond, WA, United States of America

Sumit Gulwani