International Business Machines Corporation

Yunfeng Zhang

Qingzi Vera Liao

Vera Liao

Biplav Srivastava

Elizabeth Daly

Oznur Alkan

Stephanie Houde

Rachel Katherine Emma Bellamy

Adi Ionel Botea

Akihiro Kishimoto

Radu Marinescu

John Thomas Richards

Matthew Scott Davis

Kartik Talamadupula

Bhavya Ghai

Xin Ru Wang

Chuang Gan

Dakuo Wang

Kush Raj Varshney

Rahul Nair

Yufang Hou

Michael Desmond

Massimiliano Mattetti

David John Piorkowski

Dennis Wei

Klaus Mueller

Edward Sun

David John Pirokowski

Rachel Katherine Emma Ballamy

Tian Gao

An embodiment includes identifying, from a training dataset for training a model, a first unlabeled datapoint to present for labelling according to a first query strategy. The embodiment also includes issuing a query requesting a label for the first unlabeled datapoint. The embodiment also includes receiving a labeled datapoint in response to the query, the labeled datapoint comprising the first unlabeled datapoint as labeled by an oracle. The embodiment also includes generating a causal network based on labeled datapoints from the training dataset. The embodiment also includes receiving an instruction to modify the causal network. The embodiment also includes replacing the first query strategy with a second query strategy based on the instruction to modify the causal network. The embodiment also includes identifying, from the training dataset, a second unlabeled datapoint to present for labelling according to the second query strategy.

Active learning using causal network feedback

In a method for interpreting output of a machine learning model, a processor receives a first interpretable rule set. A processor may also receive a second interpretable rule set generated from a dataset and model-predicted labels classifying the dataset. A processor may also generate a difference metric and mapping between the first interpretable rule set and the second interpretable rule set.

Interpretable model changes

An autonomous agent operating method, system, and computer program product, including launching a first autonomous agent for a task with an initial update interval and adjusting the initial update interval for a second autonomous agent based on a second task for the second autonomous agent being similar to the task over time in relation to a trust level of a human user in a performance of the first autonomous agent.

Increasing trust formation and reduce oversight costs for autonomous agents

A computer-implemented method for counterfactual conversation simulation is disclosed. The computer-implemented method includes generating a system output based, at least in part, on a user input. The computer-implemented method further includes determining that a system output/user input pair is not satisfactory based, at least in part, on a system output/user input score being below a predetermined threshold. The computer-implemented method further includes generating, in response to determining the system output/user input pair is not satisfactory, a counterfactual simulation of the user input based, at least in part, on a target intent of the user input.

Conversational agent counterfactual simulation

A method optimizes machine learning systems. A computing device accesses a committee of classifiers that have been trained using an initial labeled instance of data from an annotator. The initial labeled instance of data includes annotator-ranked attributes of the data, initial values of the attributes, and an initial prediction label that describes an initial predicted state based on the values. The computing system compares the attributes ranking from the annotator to attributes rankings that are generated by and used by each of the machine learning systems when evaluating one or more instances of unlabeled data that include the attributes, and weights the machine learning systems according to how closely each of the attributes rankings generated by and used by each of the machine learning systems match the attributes ranking from the annotator. The machine learning systems are then optimized based on this matching.

Optimizing a machine learning system

An approach is provided for training classifiers used in machine learning. A corpus of training data is received. One or more clusters of the training data is generated according to features of the training data. The one or more clusters are refined using user-specified rules. One or more classifiers are trained for use in machine learning based upon the refined one or more clusters.

Training classifiers in machine learning

Techniques that facilitate generating and executing an optimal dialogue strategy are provided. In one example, a system includes an information gain component and a question selector component. The information gain component estimates information gain data associated with first dialogue data of an information system. The information gain data is indicative of an amount of change in entropy associated with the first dialogue data. The question selector component selects between second dialogue data associated with a first question strategy and third dialogue data associated with a second question strategy based on the information gain data.

Generating and executing an optimal dialogue strategy

An autonomous agent operating method, system, and computer program product, including configuring an autonomous agent for a task, launching the autonomous agent with an initial update interval, the update interval determining a frequency of the autonomous agent reporting results to a human user for review, measuring the trust level of human user in a performance of the autonomous agent, and dynamically adjusting the update interval based on this measured trust.

Techniques regarding mitigating bias in one or more machine learning models are provided. For example, one or more embodiments described herein can comprise a system, which can comprise a memory that can store computer executable components. The system can also comprise a processor, operably coupled to the memory, and that can execute the computer executable components stored in the memory. The computer executable components can comprise a model component that can evaluate a machine learning model at a plurality of threshold settings to generate a sample set and can define a relationship between a fairness metric and a utility metric of the machine learning model based on the sample set.

Mitigating statistical bias in artificial intelligence models

Systems and methods for creating presentation slides. A slide title is received and portions of source documents relevant to the title are identified based on a dense vector information retrieval machine learning process. An abstractive summary of the portions is generated based on a long form question answering machine learning process. A first presentation slide is created with the abstractive summary and the title. The first presentation slide is presented to an operator and an input indicating one of accepting or rejection the abstractive summary is received. Based on the input that indicating rejecting the abstractive summary, the abstractive summary is removed from the presentation slide and negative training feedback for the abstractive summary is provided to at least one of the dense vector information retrieval machine learning process or the long form question answering machine learning process.

Automatic generation of presentation slides from documents

Various embodiments are provided for integrating multiple domain learning and personalization in a dialog system for a user in a computing environment by a processor. One or more problem instances may be defined for multiple domains according to a problem instance template, identified user intent, links to one or more problem solvers associated with the multiple domains, or a combination thereof. A dialog plan may be determined to further define the one or more problem instances in response to user input. A solution may be provided to the user for the one or more problem instances.

Integrating multiple domain problem solving in a dialog system for a user

Various embodiments are provided for using a dialog system for integrating multiple domain learning and problem solving for a user in a computing environment by a processor. One or more problem instances may be defined for one or more selected domains in a multi-domain database according to a problem instance template, identified user intent, links to one or more problem solvers associated with the one or more selected domains, or a combination thereof. A dialog plan may be determined for the one or more problem instances using a dialog system associated with the multi-domain database, wherein each record in the multi-domain database corresponds to a selected database for the one or more selected domains. A solution may be provided to the user for the one or more problem instances. One or more preferences of a user may be learned according to the solution.

Growing community of inventors

Tarrytown, NY, United States of America

Yunfeng Zhang