Microsoft Technology Licensing, LLC

Aparna Lakshmiratan

Carlos Garcia Jurado Suarez

David Maxwell Chickering

Patrice Y Simard

Saleema Amin Amershi

David Max Chickering

David G Grangier

Denis X Charles

Geoffrey G Zweig

Jason D Williams

Leon Bottou

Andrey Kolobov

Nobal Bikram Niraula

Pradeep Dasigi

Steven Mark Drucker

Christopher A Meek

Timothy S Paek

Bongshin Lee

Deepak Agarwal

Paul Nathan Bennett

Asela J Gunawardana

Ahmad A Abdulkader

Jason Paul Williams

Anton Mityagin

Michael David Revow

Alnur Ali

Sameer Yusufali Merchant

Patrice Yvon Rene Simard

Seyed Ali Ahmadi

A collection of data that is extremely large can be difficult to search and/or analyze. Relevance may be dramatically improved by automatically classifying queries and web pages in useful categories, and using these classification scores as relevance features. A thorough approach may require building a large number of classifiers, corresponding to the various types of information, activities, and products. Creation of classifiers and schematizers is provided on large data sets. Exercising the classifiers and schematizers on hundreds of millions of items may expose value that is inherent to the data by adding usable meta-data. Some aspects include active labeling exploration, automatic regularization and cold start, scaling with the number of items and the number of classifiers, active featuring, and segmentation and schematization.

Interactive feature selection for training a machine learning system and displaying discrepancies within the context of the document

Domain-specific language understanding models that may be built, tested and improved quickly and efficiently are provided. Methods, systems and devices are provided that enable a developer to build user intent detection models, language entity extraction models, and language entity resolution models quickly and without specialized machine learning knowledge. These models may be built and implemented via single model systems that enable the models to be built in isolation or in an end-to-end pipeline system that enables the models to be built and improved in a simultaneous manner.

Generation of language understanding systems and methods

Methods, computer systems, computer-storage media, and graphical user interfaces are provided for visualizing a performance of a machine-learned model. An interactive graphical user interface includes an item representation display area that displays a plurality of item representations corresponding to a plurality of items processed by the machine-learned model. The plurality of item representations are arranged according to scores assigned to the plurality of items by the machine-learned model. Further, each of the plurality of item representations is visually configured to represent a label assigned to a corresponding item.

Interactive visualization of machine-learning performance

Active labeling for computer-human interactive learning

Conversational interactions between humans and computer systems can be provided by a computer system that classifies an input by conversation type, and provides human authored responses for conversation types. The input classification can be performed using trained binary classifiers. Training can be performed by labeling inputs as either positive or negative examples of a conversation type. Conversational responses can be authored by the same individuals that label the inputs used in training the classifiers. In some cases, the process of training classifiers can result in a suggestion of a new conversation type, for which human authors can label inputs for a new classifier and write content for responses for that new conversation type.

Generating computer responses to social conversational inputs

Interactive segment extraction in computer-human interactive learning

Active featuring in computer-human interactive learning

Systems and methods allow an on-line game to extract information relevant to a specific need of a game platform or service platform. The specific need relates to management and use of digital content, and is addressed by designing and playing an on-line collaborative game. The rules of the game intend to solve a specific task dictated by the specific need. Players' responses to the game generate a wealth of information related to a specific task objective, such as ranking, sorting, and evaluating a set of digital content items. To compel participation in a game, players can be rewarded with monetary value rewards. As a game illustration, an image selection game (ISG) that exploits human contextual inference is described in detail. The information extracted from ISG is a list of key-image associations, relevant for the task of image sorting and ranking.

Information collection during game play

Systems, methods, and computer storage media having computer-executable instructions embodied thereon for rewriting queries and labeling word pairs. Queries are received and alternate words are identified for word pairs (i.e., query words and alternate words). Word pair links are presented to users and indicators are received based on actions taken by the users. Labels are assigned to the word pairs based on the indicators and communicated to a classifier.

Efficient label acquisition for query rewriting

Growing community of inventors

Kirkland, WA, United States of America

Aparna Lakshmiratan