Google Inc.

X Development LLC

Deepmind Technologies Limited

Rishabh Singh

David Andre

Bin Ni

Manzil Zaheer

Artem Goncharuk

Qianyu Zhang

Olivia Hatalsky

Nisarg Vyas

David J Bieber

Hanjun Dai

Kensen Shi

Simon Tong

Thomas Peter Hunt

Vishnu Sivaji

Owen Lewis

Rebecca Radkoff

Aaron Saul Zemach

Karen Roter Davis

Jonathan Matthew Malmaud

Chiraag Galaiya

Dima Brezhnev

David Lick

Francisco Velasquez

Max Lin

Neha Bhargava

Peilun Zhang

Rahul Srinivasan

Victoria Taylor

Zifan Xiao

Yujia Li

Karen E Davis

Navneet Potti

Pushmeet Kohli

Joel Galenson

Jayendra Parmar

Po-Sen Huang

Chenglong Wang

Joey Hong

Giovanni De Toni

Dhara Kotecha

This specification is generally directed to techniques for robust natural language (NL) based control of computer applications. In many implementations, the NL control is at least selectively interactive in that the user feedback input is solicited, and received, in resolving action(s), resolving action set(s), generating domain specific knowledge, and/or in providing feedback on implemented action set(s). The user feedback input can be utilized in further training of machine learning model(s) utilized in the NL based control of the computer applications.

Robust natural language based control of computer applications

Implementations are described herein for adapting existing source code snippets to new contexts. In various implementations, a command may be detected to incorporate an existing source code snippet into destination source code. An embedding may be generated based on the existing source code snippet, e.g., by processing the existing source code snippet using an encoder. The destination source code may be processed to identify one or more decoder constraints. Subject to the one or more decoder constraints, the embedding may be processed using a decoder to generate a new version of the existing source code snippet that is adapted to the destination source code.

Adapting existing source code snippets to new contexts

Implementations are described herein for predicting symbolic transformation templates to automate source code transformations. In various implementations, pair(s) of predecessor and successor source code snippets may be processed using a symbolic transformation template prediction (STTP) model to predict a symbolic transformation template that includes a predecessor portion that matches the predecessor source code snippet(s) of the pair(s) and a successor portion that matches the successor source code snippet(s) of the pair(s). At least one additional predecessor source code snippet may be identified that matches the predecessor portion of the predicted symbolic transformation template. Placeholders of the predecessor portion of the predicted symbolic transformation template may be bound to one or more tokens of the at least one additional predecessor source code snippet to create binding(s). The successor portion of the predicted symbolic transformation template may be applied to the bindings to generate additional successor source code snippet(s).

Predicting and/or applying symbolic transformation templates

Techniques are described herein for iterative code generation using neural language models. In various implementations, an original source code snippet in a first programming language may be processed using a translation machine learning model to generate a first translation of the original source code snippet in a second programming language. The first translation of the original source code snippet may be evaluated to identify error(s) in the first translation. Based on the error(s), respective mask(s) may be inserted to generate a masked first translation of the original source code snippet in the second programming language. The masked first translation of the original source code snippet may be processed using the translation machine learning model to generate a second translation of the original source code snippet in the second language. The second translation may include infill(s) of corrected source code in place of one or more of the masks.

Iterative neural code translation

Techniques are described herein for translating source code using sparse-self attention. In various implementations, a source code snippet in a first programming language may be processed to obtain graph(s) representing snippet tokens, and relationships therebetween. Based on the graph(s), a subset of snippet token pairs may be identified from a superset of all possible token pairs in the source code snippet. Each token pair of the subset may include snippet tokens that are represented by nodes connected by one or more edges of the one or more graphs. A self-attention network of a translation machine learning model may be adapted to sparsely attend across the identified subset of token pairs. The source code snippet may then be processed based on the adapted translation machine learning model to generate a translation of the source code snippet in the second programming language.

Translating large source code using sparse self- attention

Techniques are described herein for automatically synthesizing programs that include one or more functions in a spreadsheet programming language. A method includes: receiving a first example including input provided in a first cell in a spreadsheet; automatically synthesizing a plurality of candidate programs including a first set of candidate programs consistent with the first example, wherein each candidate program in the first set of candidate programs comprises at least one function in a spreadsheet programming language and, when the candidate program is executed, the candidate program generates output that matches the first example; ranking the plurality of candidate programs; and storing a highest-ranked program of the plurality of candidate programs in association with the first cell in the spreadsheet.

Synthesizing programs in a spreadsheet programming language

Techniques are described herein for translating a source code snippet from a first programming language to a second programming language independently of sequence-to-sequence decoding. In various implementations, the source code snippet written in the first programming language may be processed using an encoder portion of a transformer network to generate an embedding of the source code snippet. The embedding of the source code snippet may be processed using an all-pair attention layer to generate an attended embedding of the source code snippet. The attended embedding of the source code snippet may be processed using an output layer to generate, by way of a single transformation of the attended embedding of the source code snippet, data indicative of a translation of the source code snippet in the second programming language.

Translating between programming languages independently of sequence-to-sequence decoders

Using a natural language (NL) latent presentation in the automated conversion of source code from a base programming language (e.g., C++) to a target programming language (e.g., Python). A base-to-NL model can be used to generate an NL latent representation by processing a base source code snippet in the base programming language. Further, an NL-to-target model can be used to generate a target source code snippet in the target programming language (that is functionally equivalent to the base source code snippet), by processing the NL latent representation. In some implementations, output(s) from the NL-to-target model indicate canonical representation(s) of variables, and in generating the target source code snippet, technique(s) are used to match those canonical representation(s) to variable(s) of the base source code snippet. In some implementations, multiple candidate target source code snippets are generated, and a subset (e.g., one) is selected based on evaluation(s).

Using natural language latent representation in automated conversion of source code from base programming language to target programming language

Training and/or utilization of machine learning model(s) (e.g., neural network model(s)) in automatically generating test case(s) for source code. Techniques disclosed herein can be utilized in generating test case(s) for unit test testing (or other white-box testing) and/or for functional testing (or other black-box testing). In some implementations, the machine learning model(s) can be trained on source code, unit test pairs. In some additional or alternative implementations, reinforcement learning techniques can be utilized to check for correctness of base source code, target source code pairs (e.g., by matching program execution of different branches).

Training and/or using machine learning model(s) for automatic generation of test case(s) for source code

The present disclosure provides systems and methods for synthesizing computer-readable code based on the receipt of input and output examples. A computing system in accordance with the disclosure can be configured to receive a given input and output, access and library of operations, and perform a search of a library of operations (e.g., transpose, slice, norm, etc.) that can be applied to the input. By applying the operations to the input and tracking the results, the computing system may identify an expression comprising one or a combination of operations that when applied to the input generates the output. In this manner, implementations of the disclosure may be used to identify one or more solutions that a user having access to the library of operations may use to generate the output from the input.

Systems and methods for synthesizing code from input and output examples

Techniques are described herein for translating source code in one programming language to source code in another programming language using machine learning. A method includes: receiving first source code in a first higher-level programming language; processing the first source code, or an intermediate representation thereof, using a sequence-to-sequence neural network model to generate a sequence of outputs, each including a probability distribution; generating second source code in a second higher-level programming language by, for each output in the sequence of outputs: determining a highest probability in the probability distribution associated with the output; in response to the highest probability exceeding a first threshold, generating a predicted portion of the second source code based on a token that corresponds to the highest probability; and in response to the highest probability not exceeding the first threshold, generating a placeholder; and outputting the second source code.

Annotations for developers

Implementations are described herein for using machine learning to perform various tasks related to migrating source code based on relatively few ('few shots') demonstrations. In various implementations, an autoregressive language model may be conditioned based on demonstration tuple(s). In some implementations, a demonstration tuple may include a pre-migration version of a first source code snippet and a post-migration version of the first source code snippet. In other implementations, demonstration tuples may include other data, such as intermediate forms (e.g., natural language descriptions or pseudocode), input-output pairs demonstrating intended behavior, etc. The autoregressive language model may be trained on corpora of source code and natural language documentation on the subject of computer programming. A pre-migration version of a source code file may be processed based on the conditioned autoregressive language model, and a post-migration version may be generated based on output generated based on the conditioned autoregressive model.

Conditioning autoregressive language model to improve code migration

Growing community of inventors

San Jose, CA, United States of America

Rishabh Singh