Microsoft Technology Licensing, LLC

Margaret Ann Mitchell

Jianfeng Gao

William B Dolan

Christopher John Brockett

Michel Galley

Alessandro Sordoni

Michael Auli

Yangfeng Ji

Christopher Brian Quirk

Kristina Nikolova Toutanova

Pallavi Choudhury

Susan Michele Hendrich

Melissa Nicole Lim

Jian-Yun Nie

Mouni Reddy

Xuetao Yin

Rebecca Mason

Ron Owens

Yaxiao Song

Liang Xu

Bill Dolan

Jay Banerjee

Li Deng

John Carlton Platt

Xiaodong He

Geoffrey G Zweig

Saurabh Gupta

Piali Choudhury

Rupesh Kumar Srivastava

Forrest Iandola

Hao Fang

Examples described herein provide a digital assistant crafting a response based on target sentiment identification from user input. The digital assistant receives unstructured data input and identifies a segment of the input that includes a facet item. A sentiment associated with the facet item in the segment is identified and classified to identify a targeted sentiment directed towards the facet item. A response is generated based on the targeted sentiment and the facet item.

Crafting a response based on sentiment identification

A 'Facet Recommender' creates conversational recommendations for facets of particular conversational topics, and optionally for things associated with those facets, from consumer reviews or other social media content. The Facet Recommender applies a machine-learned facet model and optional sentiment-model, to identify facets associated with spans or segments of the content and to determine neutral, positive, or negative consumer sentiment associated with those facets and, optionally, things associated with those facets. These facets are selected by the facet model from a list or set of manually defined or machine-learned facets for particular conversational topic types. The Facet Recommender then generates new conversational utterances (i.e., short neutral, positive or negative suggestions) about particular facets based on the sentiments associated with those facets. In various implementations, utterances are fit to one or more predefined conversational frameworks. Further, responses or suggestions provided as utterances may be personalized to individual users.

Sentiment-based recommendations as a function of grounding factors associated with a user

Examples are generally directed towards context-sensitive generation of conversational responses. Context-message-response n-tuples are extracted from at least one source of conversational data to generate a set of training context-message-response n-tuples. A response generation engine is trained on the set of training context-message-response n-tuples. The trained response generation engine automatically generates a context-sensitive response based on a user generated input message and conversational context data. A digital assistant utilizes the trained response generation engine to generate context-sensitive, natural language responses that are pertinent to user queries.

Context-sensitive generation of conversational responses

Examples described herein dynamically personalize a digital assistant for a specific user, creating a personal connection between the digital assistant and the user. The digital assistant accesses user activity and generates queries based on the user activity. The digital assistant facilitates natural language conversations as machine learning sessions between the digital assistant and the user using the one or more queries to learn the user's preferences and receives user input from the user during the learning session in response to the queries. The digital assistant dynamically updates a personalized profile for the user based on the user input during the natural language conversations.

Crafting feedback dialogue with a digital assistant

Facet recommendations from sentiment-bearing content

Examples are generally directed towards automatic assessment of machine generated conversational responses. Context-message-response n-tuples are extracted from at least one source of conversational data to generate a set of multi-reference responses. A response in the set of multi-reference responses includes it context-message data pair and rating. The rating indicates a quality of the response relative to the context-message data pair. A response assessment engine generates a metric score for a machine-generated response based on an assessment metric and the set of multi-reference responses. The metric score indicates a quality of the machine-generated conversational response relative to a user-generated message and a context of the user-generated message. A response generation system of a computing device, such as a digital assistant, is optimized and adjusted based on the metric score to improve the accuracy, quality, and relevance of responses output to the user.

Metric for automatic assessment of conversational responses

Disclosed herein are technologies directed to discovering semantic similarities between images and text, which can include performing image search using a textual query, performing text search using an image as a query, and/or generating captions for images using a caption generator. A semantic similarity framework can include a caption generator and can be based on a deep multimodal similar model. The deep multimodal similarity model can receive sentences and determine the relevancy of the sentences based on similarity of text vectors generated for one or more sentences to an image vector generated for an image. The text vectors and the image vector can be mapped in a semantic space, and their relevance can be determined based at least in part on the mapping. The sentence associated with the text vector determined to be the most relevant can be output as a caption for the image.

Discovery of semantic similarities between images and text

Growing community of inventors

Seattle, WA, United States of America

Margaret Ann Mitchell