Microsoft Technology Licensing, LLC

Rishabh Singh

Sumit Gulwani

Benjamin Zorn

Dany Rouhana

Pushmeet Kohli

Oleksandr Polozov

Emilio Parisotto

Abdelrahman S A Mohamed

Ramarathnam R Venkatesan

Eric Cohen

Chad B Rothschiller

Daniel C Battagin

Vu Minh Le

Dengyong Zhou

Patrice Godefroid

Pallavi Choudhury

Mark Marron

Lihong Li

Marc Manuel Johannes Brockschmidt

William Blum

Weide Zhong

Saswat Padhi

Ben Livshits

Christopher Benedict

Gustavo Araujo Soares

Maxim Grechkin

Mikael Mayer

Dmitri O Danilov

Rodrigo Moreira-Silveira

Mohit Rajpal

Dana Drachsler Cohen

Jeevana Priya Inala

Hila Peleg

Systems, methods, and computer-readable storage devices are disclosed for improved table identification in a spreadsheet. One method including: receiving a spreadsheet including at least one table; identifying, using machine learning, one or more classes of a plurality of classes for each cell of the received spreadsheet, wherein the plurality of classes include corners and not-a-corner; and inducing at least one table in the received spreadsheet based on the one or more identified classes for each cell of the received spreadsheet.

Systems, methods, and computer-readable media for improved table identification using a neural network

A device includes a logic machine and a data-holding machine having instructions executable by the logic machine to receive a spreadsheet including a plurality of cells, apply an abstraction to the spreadsheet that defines one or more features of a cell set including one or more cells of the plurality of cells to form an abstracted representation of the spreadsheet, form, for the cell set, an input vector for a machine-learning prediction function from the abstracted representation of the spreadsheet, the machine-learning prediction function configured to output a prediction of one or more properties of the cell set based on the input vector, wherein the machine-learning prediction function is previously trained based on a plurality of previously-created spreadsheets, provide the input vector to the machine-learning prediction function; and output the prediction from the machine-learning prediction function.

Predicting spreadsheet properties

Provided are methods and systems for automatically generating input grammars for grammar-based fuzzing by utilizing machine-learning techniques and sample inputs. Neural-network-based statistical learning techniques are used for the automatic generation of input grammars. Recurrent neural networks are used for learning a statistical input model that is also generative in that the model is used to generate new inputs based on the probability distribution of the learnt model.

Machine learning for input fuzzing

Generally discussed herein are devices, systems, and methods for encoding input-output examples. A method of generating a program using an encoding of input-output examples, may include processing an input example of the input-output examples, using a first long short term memory (LSTM) neural network, one character at a time to produce an input feature vector, processing an output example associated with the input example in the input-output examples, using the LSTM neural network, one character at a time to produce an output feature vector, determining (a) a cross-correlation between the input feature vector and the output feature vector or (b) previously computed feature vectors for a different input-output example that are sufficiently close to the input feature vector and the output feature vector, respectively, and using the determined cross-correlation or previously computed vector, generating a program consistent with the input example and the output example.

Input-output example encoding

Described are systems, methods, and computer-readable media for program generation in a domain-specific language based on input-output examples. In accordance with various embodiments, a neural-network-based program generation model conditioned on an encoded set of input-output examples is used to generate a program tree by iteratively expanding a partial program tree, beginning with a root node and ending when all leaf nodes are terminal.

Neural network for program synthesis

Provided are methods and systems for joining semi-structured data from the web with relational data in a spreadsheet table using input-output examples. A first sub-task performed by the system learns a string transformation program to transform input rows of a table to URL strings that correspond to the webpages where the relevant data is present. A second sub-task learns a program in a rich web data extraction language to extract desired data from the webpage given the example extractions. Hierarchical search and input-driven ranking are used to efficiently learn the programs using few input-output examples. The learnt programs are then run on the remaining spreadsheet entries to join desired data from the corresponding web pages.

Joining web data with spreadsheet data using examples

Techniques are disclosed which provide for transforming a hierarchical table to a relational table. A hierarchical table may be received, in which a headline row is identified. A candidate row may be determined in the hierarchical table. The process may include systematically classifying headlines as data headlines or descriptors. For each data headline a new column may be generated, while for each descriptor headline, the table may be split to produce a resultant table. The resultant table may be stored and the process may be repeated until there are no headlines left to be classified. The steps performed by the system to transform the table can then be displayed on a user device using a program in the Domain-specific language, which can then be further inspected or modified to perform the desired table transformation.

Automatically converting spreadsheet tables to relational tables

Embodiments disclosed herein are related to systems and methods for using input logical patterns to generate one or more programs by an underlying Program-By-Example (PBE) system based on user input examples. A system includes a processor and a system memory. The system access a set of input data. The system receives one or more user input examples for the set of input data. The user input examples are indicative of an output that should be achieved to comply with a user determined result. The system analyzes the set of input data to identify one or more logical patterns that are common to the set of input data. The system generates one or more programs which will output the user determined result, based on a set of the one or more logical patterns that are consistent with the one or more user input examples.

Learning from input patterns in Programing-By-Example

Techniques for constrained mutation-based fuzzing are described. Machine accesses an input file of code for testing. Machine performs multiple runs of a fuzzing algorithm using the input file and the code. Each run includes: performing a mutation of one or more bytes of the input file and determining which parts of the code were executed when the code was run with the mutated input file. Machine stores, for each run, an indication of whether the mutation caused execution of a portion of the code which was not executed prior to the mutation, Machine generates heatmap of the input file based on the stored indications. The heatmap maps each of the bytes in the input file to a value indicating whether the mutation of the byte caused execution of the portion of the code for testing which was not executed prior to the mutation. Machine tailors fuzzing algorithm based on heatmap.

Machine learning for constrained mutation-based fuzz testing

Data formatting rules to convert data from one form to another form are automatically determined based on a user's edits. A machine learning heuristic is applied to a user's edits to determine a data formatting rule that may be applied to data. For example, a user may make edits that add/remove characters from data, concatenate data, extract data, rename data, and the like. The machine learning heuristic may be automatically triggered in response to an event (e.g. after a predetermined number of edits are made to a same type of data) or manually triggered (e.g. selecting a user interface option). The data formatting rule may be applied to other data and the results of the formatting reviewable by the user. Based on further edits/reviews, the data formatting rule may be updated. The data formatting rules may be stored for later use.

Formatting data by example

Systems and methods for increasing user confidence in results that are produced by one or more programs that are generated by an underlying Programming-By-Example (PBE) system based on user input examples. A plurality of generated programs that have been generated using one or more user input examples that are indicative of an output that should be achieved to comply with a user determined result are received. The generated programs are narrowed based on one or more sub-expressions of the programs that are likely to cause the resultant program to comply with the user determined result. The one or more sub-expressions are exposed. Input that selects at least one of the one or more exposed sub-expressions to thereby identify the one of the generated programs that will result in the user determined result is received.

User interaction models for disambiguation in programming-by-example

A program creation system is described which generates sets of subprograms for respective input-output examples. The program creation system then groups the sets into partitions by performing an intersection operation. According to one aspect, the program creation system generates subprograms so as to exclude tokens that are not represented by the input strings of the input-output examples. According to another aspect, the program creation system first generates the subprograms without attempting to generate loop-type expressions. If this operation produces unsatisfactory results, the program creation system repeats its processing, this time including loop-type expressions. According to another aspect, the program creation system performs the grouping operation using an expedited graph-intersection operation. According to another aspect, the program creation system ranks programs (which are created based on the results of the grouping operation) based on the presence of preferred features found in the programs.

Expedited techniques for generating string manipulation programs

Ranking technique embodiments are presented that use statistical and machine learning techniques to learn the desired ranking function for use in inductive program synthesis for the domain of string transformations. This generally involves automatically creating a training dataset of positive and negative examples from a given set of training tasks, each including multiple input-output examples. From the training dataset, a ranking function is learned that assigns an expression in a program in the domain specific language to a likelihood measure. This ranking function is then used to compute likelihoods of learnt programs from a very small number of input-output examples for a new task.

Ranking for inductive synthesis of string transformations

Semantic entity manipulation technique embodiments are presented that generate a probabilistic program capable of manipulating character strings representing semantic entities based on input-output examples. The program can then be used to produce a desired output consistent with the input-output examples from inputs of a type included in the examples. The probabilistic program is generated based on the output of parsing, transform and formatting modules. The parsing module employs a probabilistic approach to parsing the input-output examples. The transform module identifies a weighted set of transforms that are capable of producing the output item from the input items of an input-output example to a likelihood specified by their assigned weight. The formatting module generates formatting instructions that transform selected output parts into a form specified by the output items in the input-output examples.

Semantic entity manipulation using input-output examples

Inductive synthesis and combination framework technique embodiments are presented that generally perform string transformations involving lookup operations in one or more relational tables, either alone or in combination with other non-lookup operations. More particularly, a semantic string lookup transformation language is presented, which can be used to generate an inductive synthesis procedure that synthesizes a set of transformations involving lookup operations that are consistent with the given set of input-output examples. In addition, a combination framework for combining the lookup transformation language and its synthesis procedure, with other transformation languages and their associated synthesis procedures, is presented. The resulting combined synthesis procedures enable the combination framework to synthesize transformations on a rich variety of data-types.

Growing community of inventors

Kirkland, WA, United States of America

Rishabh Singh