Tableau Software, Inc.

Nokia Corporation

Nokia Technologies Oy

Salesforce, Inc.

Mitsubishi Electric Research Laboratories, Inc.

Core Wireless Licensing S.a.r.l.

Conversant Wireless Licensing S.a R.l.

Sisvel International S.a.

Vidya Raghavan Setlur

Melanie Karla Tory

Alex Djalali

Sarah E Battersby

Angel Xuan Chang

Richard Carl Gossweiler, Iii

Md Enamul Hoque Prince

Isaac James Dykeman

Jeffrey Ericson

Scott A Sherman

Matthew Michael Brehmer

David Alexander Dearman

Raja Bose

Suresh Chitturi

Allison Anne Whilden

Vivek Shrivastava

Rodion Degtyar

Theodore Kornish

Filippos Lymperopoulos

Suyang Duan

Jonathan D Que

Dirk Karis

Eliana Leite Goldner

Sanaz Golbabaei

Ramakrishna Vedantham

Tolga Capin

Marti Hearst

Xin Zhao

Quinn Able Jacobson

Jock Douglas Mackinlay

Hiroshi Horii

Hawk Yin Pang

Arjun Srinivasan

Hawk-Yin Pang

Agathe Battestini

George Chen

Tracy Wong

Arathi Sripriya Kumar

Christoph Sachse

An Shi

Miska Matias Hannuksela

Ramesh Raskar

Lu Luo

Cynthia Kuo

Michael Ingrassia

Michael Lee Gleicher

Joseph Nathaniel Kaye

Dennis Nathan Bromley

David Dearman

Aditeya Pandey

Hao Zhuang

Christopher Shawn Suley

Alexander Elias Bendeck

Antonio Haro

Dae Hyun Kim

Grant McKenzie

Haeyong Chung

Tarik Crnovrsanin

Daidi Zhong

Saeko Takagi

Bruce Gooch

Vivek Vishal Shrivastava

A method uses natural language processing for visual analysis of a dataset, in a messaging application. The method includes receiving a first natural language (NL) input directed to a data source, in a messaging application. The first NL input includes at least one underspecified or ambiguous utterance. The method also includes parsing the first NL input into tokens based on a grammar and the data source. The method also includes generating and displaying an intermediate NL response, based on the tokens. In response to receiving a user input to provide missing information in the at least one underspecified or ambiguous utterance, the method includes generating an input query based on the user input and querying the data source using the input query, to obtain a result set. The method also includes generating and displaying a first NL output and a snapshot of a data visualization based on the result set.

Using messaging system threading for interpreting natural language analytical conversations

A computing device parses a user-specified natural language command to form a first expression. The computing device determines that the first expression is ambiguous or underspecified. The computing device, in accordance with the determination, infers first information using one or more inferencing rules, where at least one of the inferencing rules is based on an attribute of data fields and/or data values in a data source. The computing device forms a second expression based on the first expression using and the first information. The computing device retrieves one or more data sets from the data source using according to the second expression. The computing device generates and displays a data visualization of the retrieved one or more data sets.

Analyzing underspecified natural language utterances in a data visualization user interface

A computing device receives a natural language command directed to a data source. The device applies one or more respective rules to determine whether the natural language command comprises an ambiguous and/or underspecified request. In accordance with a determination that the natural language command comprises an ambiguous and/or underspecified request, the device infers information to resolve the ambiguous and/or underspecified request according to (i) the one or more respective rules, (ii) metadata for the data source, and (iii) metadata for one or more data fields specified in the natural language command. The device updates a respective intent specification based on the inferred information, retrieving data sets from the data source according to the respective intent specification, and generates and displays a data visualization based on the retrieved data sets.

Applying a visual analytics intent language to generate data visualizations

A computer system receives a natural language input directed to a data source. In response to receiving the natural language input, and in accordance with a determination that the natural language input specifies an incomplete natural language command directed to the data source, the computer system presents a map widget for selecting geospatial data points from the data source. The map widget includes a map having a plurality of predetermined geographic regions. The computer system receives user specification of a user-defined region in the map included in the map widget. In accordance with receiving the user specification and based on a coverage metric computed for the plurality of geographic regions, the computer system selects one or more geographic regions. The computer system completes the natural language command with the selected geographic regions. The computer system generates and displays a map data visualization according to the completed natural language command.

Visual autocompletion for geospatial queries

A plurality of interrelated block objects are obtained. Each block object having a respective specification that includes (i) a layout type; (ii) a mark type; (iii) one or more visual encodings; and (iv) one or more data fields of a data source. A respective data field of the one or more data fields is associated with a respective visual encoding of the one or more visual encodings. A respective query generated for each block object of the plurality of block objects is executed to retrieve a respective set of tuples from the data source. Each tuple comprising data values for the respective one or more data fields of the block object. A data visualization is generated in accordance with the respective specifications of the interrelated block objects. Each tuple in each of the respective set of tuples is represented as a visual mark in the data visualization.

Data visualization analytical canvas with functionally independent visual containers

Embodiments are directed to intent driven dashboard recommendations. A plurality of collection specifications that declare visualization declarations may be provided such that each collection specification is associated with an author intent. And attributes-of-interest selected from a data source may be provided. A plurality of collections may be generated based on the plurality of collection specifications, the attributes-of-interest, or data from the data source such that each collection may include visualizations that may be based on the visualization declarations. A preference score may be generated for each collection based on the author intent, visualizations in each collection, or the attributes-of-interest. The plurality of collections may be classified based on the preference scores associated with the classified collections such that the classified collections may be displayed based on the preference scores.

Intent driven dashboard recommendations

A method uses natural language processing for visual analysis of a dataset. The method includes receiving a first natural language (NL) input directed to a data source, from a first client. The method also includes parsing the first NL input into tokens based on a grammar and the data source. The method also includes generating and outputting an intermediate NL response, to a second client, based on the tokens and output modality of the second client. In response to receiving a user input to provide missing information: the method also includes generating an input query based on the user input; and querying the data source using the input query, to obtain a result set. The method also includes generating and outputting, to the second client, a first NL output and a snapshot of a data visualization, based on the result set and the output modality of the second client.

Optimizing natural language analytical conversations using platform-specific input and output interface functionality

A computing device receives a natural language input specifying search terms directed to a dataset. The device parses the input into tokens and executes queries against a search index to retrieve a plurality of labeled trend events. Each labeled trend event has a respective chart identifier. The device determines a respective composite score for each labeled trend event and individually assigns each labeled trend event to a respective group. For each group, the device sorts the respective labeled trend events within the respective group according to respective composite scores, determines a respective final score for each group, and ranks the groups according to one or more determined final scores. The device retrieves data corresponding to a first subset of line charts having the respective chart identifiers of the ranked groups, generates the first subset of line charts, and displays one or more line charts of the first subset with annotation.

Search tool for exploring quantifiable trends in line charts

A computing device receives user selection of a data source. In accordance with the user selection, the device generates one or more initial natural language utterances according to metrics of data fields in the data source and/or previous user interaction with the data source. Each of the initial natural language utterances corresponds to a respective suggestion to guide visual analysis of the data source. The device displays the initial utterances in a graphical user interface. The device receives user selection of a first initial utterance of the initial utterances. In response to the user selection, the device generates and displays a first data visualization in accordance with data fields and/or analytical operations specified in the first initial utterance. The device also generates updated natural language utterances in accordance with the first initial utterance and the first data visualization, and displays the updated utterances with the first data visualization.

Contextual utterance recommendations for natural language interfaces that support conversational visual analysis

A method executes at a computing device that includes a display, one or more processors, and memory. The device displays a data visualization based on a dataset retrieved from a database. The device also displays one or more first phrases in a first region. The first phrases define the data visualization. The device receives a first user input in a second region to specify a natural language command related to the displayed data visualization. In response to the first user input, the device displays one or more proposed actions. The device receives user selection of a first proposed action of the proposed actions. In response to the user selection, the device generates an updated data visualization. The device displays the updated data visualization and displays a plurality of second phrases in the first region. The second phrases define the updated data visualization.

Using refinement widgets for data fields referenced by natural language expressions in a data visualization user interface

An electronic device has a first device type. The device receives a request directed to a data source. The request includes one or more intent types and one or more predefined attributes associated with the one or more intent types. The device formulates, for each of the intent types, a respective intent specification according to the request. In some implementations, the device determines respective properties of the respective intent specification according to the respective predefined attributes and according to data fields, from the data source, specified in the request. The device generates, for each of the intent types, a respective output specification that specifies a respective recommended data visualization type. The device renders data analysis responsive to the request in accordance with the electronic device having the first device type.

A computing device displays a data visualization interface and receives user selection of a data source and a natural language command directed to the data source. The device forms a first intermediate expression according to a context-free grammar and a semantic model of data fields in the data source. In accordance with a determination that the first intermediate expression omits sufficient information for generating a data visualization, the device infers the omitted information associated with the data source using one or more inferencing rules based on syntactic and semantic constraints imposed by the context-free grammar. The device forms an updated intermediate expression, and translates the updated intermediate expression into database queries. It executes the database queries to retrieve data sets from the data source, then generates and displays a data visualization of the retrieved data sets.

A method uses natural language for visual analysis of a dataset. A data visualization application displays a data visualization, at a computer, based on a dataset retrieved from a database using a set of one or more queries. A user specifies a natural language command related to the displayed data visualization, and the computer extracts an analytic phrase from the natural language command. The computer computes semantic relatedness between the analytic phrase and numeric data fields in the dataset. The computer identifies numeric data fields having highest semantic relatedness to the analytic phrase, and also selects a relevant numerical function. The numerical function compares data values in the numeric data fields to a threshold value. The computer retrieves an updated dataset that filters the identified numeric data fields according to the numeric function. The computer then displays an updated data visualization using the updated dataset.

Applying natural language pragmatics in a data visualization user interface

At a computer system a graphical user interface is displayed on a display. The graphical user interface includes (a) a schema region displaying data field icons, each data field icon representing a respective data field from a user-selected data source, and (b) a canvas region. In response to receiving user input selecting a data field icon from the schema region and placing it on the canvas region, a block card is displayed on the canvas region. The block card includes user interface affordances to designate (i) a layout type that specifies a row-based arrangement, a column-based arrangement, or an inline arrangement, (ii) a mark type that specifies a shape for visual data marks, and (iii) one or more visual encodings. A data visualization for the data source is generated and displayed according to the layout type, the mark type, and the visual encodings specified on the block card.

A method uses natural language for visual analysis of a dataset, and is performed at a computer having a display, processors, and memory. The computer displays a data visualization based on a dataset retrieved from a database and receives a first user input to specify a natural language command related to the displayed data visualization. Based on the displayed data visualization and the natural language command, the computer identifies an initial range of values for a first data field in the dataset and displays an editable user interface control showing the initial range of values. The computer receives a second user input, at the user interface control, specifying a modified range of values for the first data field and retrieves an updated dataset from the database using the modified range of values for the first data field. The computer then updates the displayed data visualization using the updated dataset.

Using natural language processing for visual analysis of a data set

Growing community of inventors

Portola Valley, CA, United States of America

Vidya Raghavan Setlur