Tableau Software, Inc.

Melanie Karla Tory

Vidya Raghavan Setlur

Richard Carl Gossweiler, Iii

Sarah E Battersby

Angel Xuan Chang

Isaac James Dykeman

Md Enamul Hoque Prince

Alex Djalali

Marti Hearst

A method uses natural language processing for visual analysis of a dataset. The method includes receiving a first natural language (NL) input directed to a data source, from a first client. The method also includes parsing the first NL input into tokens based on a grammar and the data source. The method also includes generating and outputting an intermediate NL response, to a second client, based on the tokens and output modality of the second client. In response to receiving a user input to provide missing information: the method also includes generating an input query based on the user input; and querying the data source using the input query, to obtain a result set. The method also includes generating and outputting, to the second client, a first NL output and a snapshot of a data visualization, based on the result set and the output modality of the second client.

Optimizing natural language analytical conversations using platform-specific input and output interface functionality

A method uses natural language for visual analysis of a dataset. A data visualization application displays a data visualization, at a computer, based on a dataset retrieved from a database using a set of one or more queries. A user specifies a natural language command related to the displayed data visualization, and the computer extracts an analytic phrase from the natural language command. The computer computes semantic relatedness between the analytic phrase and numeric data fields in the dataset. The computer identifies numeric data fields having highest semantic relatedness to the analytic phrase, and also selects a relevant numerical function. The numerical function compares data values in the numeric data fields to a threshold value. The computer retrieves an updated dataset that filters the identified numeric data fields according to the numeric function. The computer then displays an updated data visualization using the updated dataset.

Applying natural language pragmatics in a data visualization user interface

A method incorporates data visualization into conversational interfaces. The method receives a user input specifying a natural language command via a conversational interface. The method analyzes the natural language command to determine the type of question. The method also obtains a user preference for viewing responses based on text and/or visualizations. When the user preference includes visualizations and the type of question is answerable using data visualizations, the method: (i) extracts one or more independent analytic phrases from the natural language command; (ii) queries a database using a set of queries based on the extracted analytic phrases, thereby retrieving a data set; and (iii) generates and displays, in the conversational interface, a response incorporating one or more data visualizations, based on the type of question, using the data set.

Incorporating data visualizations into database conversational interfaces

A method uses natural language for visual analysis of a dataset, and is performed at a computer having a display, processors, and memory. The computer displays a data visualization based on a dataset retrieved from a database and receives a first user input to specify a natural language command related to the displayed data visualization. Based on the displayed data visualization and the natural language command, the computer identifies an initial range of values for a first data field in the dataset and displays an editable user interface control showing the initial range of values. The computer receives a second user input, at the user interface control, specifying a modified range of values for the first data field and retrieves an updated dataset from the database using the modified range of values for the first data field. The computer then updates the displayed data visualization using the updated dataset.

Using natural language processing for visual analysis of a data set

A method uses natural language processing for visual analysis of a dataset by a computer. The computer displays a data visualization based on a dataset retrieved from a database. The computer computes an initial visualization state that includes elements corresponding to data attributes of the data visualization. The computer receives user input to specify a natural language command related to the displayed data visualization. The computer extracts cue phrases from the natural language command. The computer also determines a transitional goal, to transform the data visualization, based on the cue phrases. The computer derives an updated visualization state based on the transitional goal, by applying transitional rules to each element of the initial visualization state. The computer subsequently computes analytical functions associated with the visualization states, thereby creating functional phrases. The computer then updates the data visualization based on the functional phrases.

Identifying intent in visual analytical conversations

A computing device displays an initial data visualization according to an initial visual specification that specifies a data source, visual variables, and data fields from the data source. The computing device receives a natural language command that includes a request for information from the data source. The computing device extracts one or more first keywords and determines, based on the first keywords and one or more of (i) the data source, (ii) the visual variables, and/or (iii) the data fields of the initial visual specification, that the request does not directly specify a characteristic in the initial visual specification. The computing device generates a modified visual specification. The computing device generates one or more queries based on the modified visual specification. The computing device executes the one or more queries to retrieve data for a modified data visualization. The computing device generates and displays the modified data visualization.

Inferring intent and utilizing context for natural language expressions in a data visualization user interface

A computing device receives a user input to specify a natural language command directed to a data source. The device identifies a first keyword, corresponding to a first data field from the data source, in the natural language command. The device identifies a second keyword in the natural language command. The second keyword comprises an adjective that provides context for the first data field. The device generates a visual specification based on the first and second keywords. The visual specification specifies the data source, a plurality of visual variables, and a plurality of data fields from the data source, including the first data field. The visual variables include a first visual variable that specifies filtering of the first data field according to the second keyword. The device generates and displays a data visualization based on the visual specification.

Handling vague modifiers in natural language commands

A computing device displays an initial data visualization according to an initial visual specification that specifies a data source, visual variables, and data fields from the data source. The computing device receives a natural language command that includes a user request for information from the data source. The computing device identifies one or more keywords from the natural language command and determines, based on the keywords and one or more of (i) the data source, (ii) the visual variables, and/or (iii) the data fields of the initial visual specification, that the request indirectly specifies a first visual characteristic in the initial visual specification. The computing device generates one or more queries in accordance with the determining. The computing device executes the queries to retrieve data for a modified data visualization. The computing device generates and displays the modified data visualization.

A computing device receives user selection of a data source and a natural language command directed to the data source. The device identifies a first keyword and a second keyword in the natural language command. The first keyword corresponds to a first data field from the data source and the second keyword expresses a limit on a range of data values for the first data field. The device generates a visual specification that specifies the data source, a plurality of visual variables, and a plurality of data fields from the data source, including the first data field. The visual variables determine characteristics of visual marks in a data visualization according to the second keyword, and each of the visual variables is associated with a respective data field of the plurality of data fields. The device generates and displays the data visualization based on the visual specification.

Determining ranges for vague modifiers in natural language commands

A method modifies data visualizations based on user selected data sources and user input that specifies natural language commands requesting information from the data sources. The computer displays an initial data visualization according to a visual specification that specifies a plurality of visual variables. The computer determines one or more keywords from the natural language command and determines, based on the one or more keywords, a user intent to modify the data visualization. The computer modifies the plurality of visual variables based on the first user intent and displays a modified data visualization in accordance with the modified plurality of visual variables.

Methods and systems for inferring intent and utilizing context for natural language expressions to modify data visualizations in a data visualization interface

A computing device displays a data visualization interface and receives user selection of a data source and a natural language command directed to the data source. The device identifies a first keyword and second keyword(s) in the natural language command. The second keyword(s) include adjectives that modify the first keyword. The device generates a visual specification that specifies the data source, a plurality of visual variables, and a plurality of data fields from the data source. Each of the visual variables is associated with a respective data field of the plurality of data fields. The first keyword corresponds to one or more first data fields of the plurality of data fields, and the one or more visual variables are associated with the one or more first data fields according to the one or more second keywords. The device generates and displays a data visualization based on the visual specification.

A method generates data visualizations based on user selected data sources and user input that specifies natural language commands requesting information about the data sources. The computer determines one or more keywords from the natural language command and determines, based on the one or more keywords, a user intent to generate a new data visualization. The computer then generates a visual specification that specifies a plurality of visual variables. Each visual variable of the plurality of visual variables is generated based on the first user intent. The computer then generates and displays a data visualization based on the visual specification.

Methods and systems for inferring intent and utilizing context for natural language expressions to generate data visualizations in a data visualization interface

A method uses natural language for visual analysis of a dataset. The method displays a data visualization based on a first dataset. The method then extracts analytic phrases from a first natural language command related to the data visualization. The method computes conversation centers associated with the first natural language command based on the analytic phrases and computes analytical functions for the conversation centers, thereby creating functional phrases. The method updates the data visualization based on the functional phrases. The method extracts new analytic phrases from a second natural language command related to the updated data visualization and computes a temporary conversation centers associated with the second natural language command based on the new analytic phrases. The method derives new conversation centers from the original conversation centers and the temporary conversation centers using transitional rules. The method then updates the data visualization based on the new conversation centers.

Updating displayed data visualizations according to identified conversation centers in natural language commands

A method uses natural language for visual analysis of a dataset. A data visualization is displayed based on a first dataset. The method then extracts analytic phrases from a natural language command related to the data visualization. The method computes conversation centers based on the analytic phrases and computes analytical functions associated with the conversation centers, thereby creating functional phrases. The method updates the data visualization according to the functional phrases. The method then extracts analytic phrases from a second natural language command related to the updated data visualization, and computes temporary conversation centers from these analytic phrases. The method then computes cohesion between the first analytic phrases and the second analytic phrases to build a set of conversation centers, and computes analytical functions from this set of conversation centers, thereby creating functional phrases. The method updates the data visualization based on the created functional phrases.

Growing community of inventors

Palo Alto, CA, United States of America

Melanie Karla Tory