International Business Machines Corporation

Evelyn Duesterwald

Tamir Klinger

Peter Kenneth Malkin

Mark N Wegman

Paul Michael Matchen

Murray Robert Cantor

Peri L Tarr

Stanley M Sutton

Gheorghe Calin Cascaval

Liana Liyow Fong

Michael Desmond

David John Piorkowski

Wei Tan

Julian Timothy Dolby

Peter Francis Sweeney

Xiaolong Xie

Guillaume A Baudart

Anupama Murthi

Robert William Wisniewski

Heiko Hary Ludwig

Vinod Muthusamy

Darrell Christopher Reimer

Matthew R Arnold

Harold Leon Ossher

Sandhya Dwarkadas

Vatche Isahagian

Bryant R Chen

John Charles Thomas

Patrick A Wagstrom

Nathalie Baracaldo-Angel

Debashish Saha

Punleuk Oum

Gaodan Fang

Valentina Salapura

Michael S Gordon

Komminist S Weldemariam

Ying Li

Martin J Hirzel

Kevin Alexander Stoodley

Peter Hans Westerink

Allan Henry Kielstra

Yuan Yuan Zhang

Jiri Navratil

Anuradha Bhamidipaty

Vijay Arya

John Thomas Richards

Robert Leslie Yates

Maria Rene Ebling

Dharmashankar Subramanian

Avraham Ever Shinnar

Andrew Lawrence Frenkiel

Michael James Hind

Vijay Anand Saraswat

Xiao Zhong

Qi Ming Teng

Qiming Teng

Deepak Vijaykeerthy

Benjamin W Herta

Waldemar Hummer

Siddhartha Chatterjee

Kristina Marie Brimijoin

Benjamin Tyler Elder

Haichuan Wang

Ganesh Venkataraman

Archit Verma

Robert M Fuhrer

Austin Zachary Henley

Yiyun Chen

Techniques are provided for decision making tasks using a hybrid approach where cooperation between an AI assessor and a human labeler controls automation of the process. In one aspect, a method for hybrid decision making automation includes: monitoring interactions between an AI assistant and a human decision maker; tracking, from the interactions, agreement of the human decision maker with decision predictions made by the AI assistant; determining a predicted performance of data tasks by the AI assistant on unseen data based on the agreement of the human decision maker with the decision predictions over time; and assessing delegation of remaining data tasks on the unseen data to the AI assistant using the predicted performance.

Hybrid decision making automation

A method of using a computing device to manage a lifecycle of machine learning models includes receiving, by a computing device, multiple pre-defined machine learning lifecycle tasks. The computing device manages executing a management-layer software layer for the multiple pre-defined machine learning lifecycle tasks. The computing device further generates and updates a machine learning pipeline using the management-layer software layer.

Configurable and scalable machine learning model lifecycle operator

Performance of a computer implementing a machine learning system is improved by providing, via a graphical user interface, to an annotator, unlabeled corpus data to be labeled; obtaining, via the interface, labels for the unlabeled corpus data; and detecting, with a consistency calculation routine, concurrent with the labeling, internal inconsistency and/or external inconsistency in the labeling. Responsive to the detection, intervene in the labeling with a reactive intervention subsystem until the inconsistency is addressed. The labeling is completed subsequent to the intervention; the system is trained to provide a trained machine learning system, based on results of the completed labeling; and classification of new data is carried out with the trained system. Proactive intervention schemes are also provided.

Framework of proactive and/or reactive strategies for improving labeling consistency and efficiency

A computer implemented method of performing large-scale machine learning experiments includes expanding on one or more input datasets by systematically generating several data set drift splits. A set of experimental jobs corresponding to the generated data set drift splits are executed to generate experimental results. The experimental results are processed, consolidated, and clustered according to the generated data set drift splits.

Assistant for large-scale machine learning experiments

Computer-implemented methods, program products, and systems for provenance-based defense against poison attacks are disclosed. In one approach, a method includes: receiving observations and corresponding provenance data from data sources; determining whether the observations are poisoned based on the corresponding provenance data; and removing the poisoned observation(s) from a final training dataset used to train a final prediction model. Another implementation involves provenance-based defense against poison attacks in a fully untrusted data environment. Untrusted data points are grouped according to provenance signature, and the groups are used to train learning algorithms and generate complete and filtered prediction models. The results of applying the prediction models to an evaluation dataset are compared, and poisoned data points identified where the performance of the filtered prediction model exceeds the performance of the complete prediction model. Poisoned data points are removed from the set to generate a final prediction model.

Detecting and mitigating poison attacks using data provenance

Methods and systems for multi-dimensional data labeling. A structured data set having a plurality of rows is obtained, the structured data set comprising a set of data attributes, each data attribute having a data value for each of the plurality of rows of the structured data set. The structured data set is decomposed into a plurality of dimensions, each dimension defining a proper subset of the data attributes based on coherence criterion. A dimension label is obtained for each dimension of at least a portion of the plurality of rows of the structured data set and the dimension labels for a given one of the rows of the structured data set are consolidated into at least one row label for the given one of the rows.

Multi-dimensional data labeling

Systems, computer-implemented methods, and computer program products to facilitate model transfer learning across evolving processes are provided. According to an embodiment, a system can comprise a memory that stores computer executable components and a processor that executes the computer executable components stored in the memory. The computer executable components can comprise a condition definition component that defines one or more conditions associated with use of a model trained on first traces of a first process to make a prediction on one or more second traces of a second process. The computer executable components can further comprise a guardrail component that determines whether to use the model to make the prediction.

Model transfer learning across evolving processes

A system, method, and computer program product for verifying signatures. The system includes at least one processing component, at least one memory component, and a reference storage comprising a set of reference signatures. The system also includes a model generator configured to generate a signature model based on the set of reference signatures. Further, the system includes a verification component configured to receive a signature, and determine whether the signature is valid.

Multifactor handwritten signature verification

The generating of actionable recommendations for tuning model metrics of an Artificial Intelligence (AI) system includes partitioning a key performance indicator (KPI) range associated with a target system into a plurality of buckets. Log data including at least one KPI of the target system and one or more AI model metrics is partitioned and distributed across the plurality of buckets. For each bucket, an aggregate value of the one or more AI model metrics across the log data is computed and weighted according to the volume of log data in that bucket. A correlation factor between the aggregate value and a representative KPI value for each bucket is determined. A model tuning recommendation to increase ranking of the AI model metrics according to the determined correlation factor is provided to an output device and/or to the AI system for updating the one or more AI model.

Generating model insights by progressive partitioning of log data across a set of performance indicators

In an embodiment, a method for inspecting and transforming a machine learning model includes receiving a request that includes the machine learning model and a configuration object that provides an indication of a selected strategy. In the embodiment, the method includes creating a partially specified task graph that includes a first placeholder node for a future expanded task node. In the embodiment, the method includes performing a dynamic expansion and execution phase that includes, repeatedly (a) using a cognitive engine to evaluate whether to revise the partially specified task graph based at least in part on the selected strategy, and (b) using a processor-based execution engine to perform an action specified by the complete node. In an embodiment, the dynamic expansion and execution phase repeats until after the cognitive engine adds a consolidated results node.

System for operationalizing high-level machine learning training enhancements from low-level primitives

Techniques that facilitate matrix factorization associated with graphics processing units are provided. In one example, a system includes a first graphics processing unit, a second graphics processing unit and a central processing unit. The first graphics processing unit processes a first data block of a data matrix associated with a matrix factorization system to generate first information for the matrix factorization system. The second graphics processing unit processes a first portion of a second data block of the data matrix separate from a second portion of the second data block to generate second information for the matrix factorization system. The central processing unit processes a machine learning model for the matrix factorization system based on at least the first information provided by the first graphics processing unit and the second information provided by the second graphics processing unit.

Matrix factorization with two-stage data block dispatch associated with graphics processing units

A computer-implemented method includes obtaining data associated with execution of a model deployed in a computing environment. At least a portion of the obtained data is analyzed to detect one or more failure conditions associated with the model. One or more restoration operations are executed to generate one or more restoration results to address one or more detected failure conditions. At least a portion of the one or more restoration results is sent to the computing environment in which the model is deployed.

Detection of failure conditions and restoration of deployed models in a computing environment

Aspects of the present invention disclose a method, computer program product, and system for detecting and mitigating adversarial virtual interactions. The method includes one or more processors initiating a mitigation protocol on interactions between the user and the virtual agent, wherein the mitigation protocol is based on the actions performed by the user while interacting with the virtual agent. The method further includes one or more processors, in response to initiating the mitigation protocol on interactions between the user and the virtual agent, generating a lower fidelity response from the virtual agent to the user, wherein the lower fidelity response is a progressive dilution of the precision of language of an original response from the virtual agent to the user prior to the user exceeding the risk level threshold.

Cognitive virtual detector

Embodiments include method, systems and computer program products for a path-sensitive contextual help system. In some embodiments, user actions are obtained from a user session of a user. A concrete user action trace is captured using the obtained user actions, wherein the concrete user action trace is a subset of user actions from the user session. An abstract user action trace is generated using the concrete user action trace. A help action corresponding to the abstract user action trace is identified and, in some embodiments, is presented to the user.

Path-sensitive contextual help system

Growing community of inventors

Millwood, NY, United States of America

Evelyn Duesterwald