Tableau Software, Inc.

Alex Djalali

Vidya Raghavan Setlur

Jeffrey Ericson

Melanie Karla Tory

Suyang Duan

Theodore Kornish

Eliana Leite Goldner

Sanaz Golbabaei

Xin Zhao

Ryan Andrew Atallah

Yukiko Ishida Anonuevo

Qixiang Zhang

An Shi

Christoph Sachse

Amar Singh Lakhian

Hao Zhuang

Christopher Shawn Suley

Andre Lukas Schorlemmer

A computing device parses a user-specified natural language command to form a first expression. The computing device determines that the first expression is ambiguous or underspecified. The computing device, in accordance with the determination, infers first information using one or more inferencing rules, where at least one of the inferencing rules is based on an attribute of data fields and/or data values in a data source. The computing device forms a second expression based on the first expression using and the first information. The computing device retrieves one or more data sets from the data source using according to the second expression. The computing device generates and displays a data visualization of the retrieved one or more data sets.

Analyzing underspecified natural language utterances in a data visualization user interface

A method executes at a computing device that includes a display, one or more processors, and memory. The device displays a data visualization based on a dataset retrieved from a database. The device also displays one or more first phrases in a first region. The first phrases define the data visualization. The device receives a first user input in a second region to specify a natural language command related to the displayed data visualization. In response to the first user input, the device displays one or more proposed actions. The device receives user selection of a first proposed action of the proposed actions. In response to the user selection, the device generates an updated data visualization. The device displays the updated data visualization and displays a plurality of second phrases in the first region. The second phrases define the updated data visualization.

Using refinement widgets for data fields referenced by natural language expressions in a data visualization user interface

The various implementations described herein include methods and devices for recommending phrases for data visualizations. In one aspect, a method includes presenting a data visualization page to a user, the page including a first region for displaying a data visualization and a second region for phrase recommendations. The method further includes obtaining a dataset selected by the user, the dataset including a plurality of fields; and generating a first set of phrase recommendations based on the dataset, each phrase recommendation corresponding to a respective field. The method also includes displaying the first set of phrase recommendations in the second region; and receiving a user selection of a first phrase. The method further includes, in response to the user selection: presenting a data visualization in the first region using the first phrase; and displaying a second set of phrase recommendations generated based on the first phrase.

Phrase recommendations for data visualizations

A computing device displays a data visualization interface and receives user selection of a data source and a natural language command directed to the data source. The device forms a first intermediate expression according to a context-free grammar and a semantic model of data fields in the data source. In accordance with a determination that the first intermediate expression omits sufficient information for generating a data visualization, the device infers the omitted information associated with the data source using one or more inferencing rules based on syntactic and semantic constraints imposed by the context-free grammar. The device forms an updated intermediate expression, and translates the updated intermediate expression into database queries. It executes the database queries to retrieve data sets from the data source, then generates and displays a data visualization of the retrieved data sets.

A computing device has a display, one or more processors, and memory. The memory stores one or more programs configured for execution by the one or more processors. The computing device receives, via a first region of a graphical user interface, a first natural language input related to a data source. The first natural language input includes one or more terms. In response to receiving the first natural language input, computing device generates one or more interpretations for the first natural language input, including: determining a first term of the one or more terms that is excluded from the one or more interpretations, and visually de-emphasizing the first term in the first region.

Visually correlating individual terms in natural language input to respective structured phrases representing the natural language input

The various implementations described herein include methods and devices for constructing phrases for data visualizations. In one aspect, a method includes presenting a data visualization page to a user including a visualization region and a phrase region; obtaining a dataset including a plurality of fields; displaying a phrase affordance for constructing a phrase; in response to a user selection of the phrase affordance, presenting a phrase construction menu including a set of fields; in response to a user selection of a field, presenting a set of aggregation operators selected based on a field type of the field; in response to a user selection of an aggregation operator, generating a first phrase using the selected field and the selected aggregation operator; presenting a data visualization generated using the first phrase; and displaying the first phrase as an active phrase in the phrase region.

Phrase builder for data analytic natural language interface

A computing device displays, in a graphical user interface, a data visualization and a plurality of phrases that collectively define the data visualization. The device receives user input to specify a natural language command related to the data visualization. The device determines that the natural language command includes a metacommand to: (1) add a data field to the data visualization, or (2) remove a data field from the data visualization, or (3) replace a data field in the data visualization with another data field. The device determines one or more proposed actions in accordance with the determined metacommand and ranks the proposed actions. The device receives user selection of a first proposed action. In response to the user selection, the device generates and displays an updated data visualization and one or more updated phrases that collectively define the updated data visualization.

Incremental updates to natural language expressions in a data visualization user interface

A computing device displays an initial data visualization according to an initial visual specification that specifies a data source, visual variables, and data fields from the data source. The computing device receives a natural language command that includes a request for information from the data source. The computing device extracts one or more first keywords and determines, based on the first keywords and one or more of (i) the data source, (ii) the visual variables, and/or (iii) the data fields of the initial visual specification, that the request does not directly specify a characteristic in the initial visual specification. The computing device generates a modified visual specification. The computing device generates one or more queries based on the modified visual specification. The computing device executes the one or more queries to retrieve data for a modified data visualization. The computing device generates and displays the modified data visualization.

Inferring intent and utilizing context for natural language expressions in a data visualization user interface

A computing device displays an initial data visualization according to an initial visual specification that specifies a data source, visual variables, and data fields from the data source. The computing device receives a natural language command that includes a user request for information from the data source. The computing device identifies one or more keywords from the natural language command and determines, based on the keywords and one or more of (i) the data source, (ii) the visual variables, and/or (iii) the data fields of the initial visual specification, that the request indirectly specifies a first visual characteristic in the initial visual specification. The computing device generates one or more queries in accordance with the determining. The computing device executes the queries to retrieve data for a modified data visualization. The computing device generates and displays the modified data visualization.

A computing device receives user input to specify a natural language command directed to a data source. In accordance with the user input, the device forms an intermediate expression according to a context-free grammar and a semantic model of data fields in the data source. The natural language command includes (i) a first term that specifies an aggregation type in a first aggregation, (ii) a second term that specifies a data field, in the semantic model, to be aggregated for the first aggregation, and (iii) terms that specify data fields, in the semantic model, to determine grouping for the first aggregation. The device translates the intermediate expression into database queries, executes the database queries to retrieve one or more data sets from the data source, aggregated according to the first aggregation, then generates and displays a data visualization of the retrieved data sets.

Using natural language constructs for data visualizations

A method executes at a computing device that includes a display, one or more processors, and memory. The method includes receiving user input to specify a data source. The method includes receiving a first user input in a first region of a graphical user interface to specify a natural language command related to the data source. The device determines, based on the first user input, that the natural language command includes a table calculation expression. In accordance with the determination, the method identifies a second data field in the data source, Values of the first data field are aggregated for each of the time periods in a range of dates according to the second data field. A respective difference between the aggregated values for each consecutive pair of time periods is computed. A data visualization is generated and displayed.

Using natural language expressions to define data visualization calculations that span across multiple rows of data from a database

A method is performed at a computing device that includes a display, one or more processors, and memory. The device displays a data visualization based on a dataset retrieved from a database. It also displays one or more phrases in a first region. The phrases define the data visualization. The device receives user input in a second region to specify a natural language command related to the displayed data visualization. In response to the user input, the device extracts independent analytic phrases from the command, determines one or more proposed actions based on the extracted phrases, and displays the proposed actions. The device receives user selection of a first proposed action. In response, the device generates an updated data visualization. It displays the updated data visualization and displays one or more updated phrases in the first region. The updated phrases define the updated data visualization.

A method modifies data visualizations based on user selected data sources and user input that specifies natural language commands requesting information from the data sources. The computer displays an initial data visualization according to a visual specification that specifies a plurality of visual variables. The computer determines one or more keywords from the natural language command and determines, based on the one or more keywords, a user intent to modify the data visualization. The computer modifies the plurality of visual variables based on the first user intent and displays a modified data visualization in accordance with the modified plurality of visual variables.

Methods and systems for inferring intent and utilizing context for natural language expressions to modify data visualizations in a data visualization interface

A computing device has a display, one or more processors, and memory. The memory stores one or more programs configured for execution by the one or more processors. The computing device receives from a user a partial natural language input related to a data source. The partial natural language input includes a most recently entered first term. In response to receiving the first term, the computing device generates a first token that includes the first term. The computing device maps the first token to one or more analytical concepts in a lexicon of the data source. The computing device determines a first interpretation corresponding to the first token. The computing device also displays a first phrase corresponding to the first interpretation. The first interpretation includes the first term.

Growing community of inventors

Los Gatos, CA, United States of America

Alex Djalali