International Business Machines Corporation

Madhavi Katari

Manish Anand Bhide

Ravi Chandra Chamarthy

Travis Eli Nelson

Sarbajit Kumar Rakshit

Kuntal Dey

Seema Nagar

Diptikalyan Saha

Pranay Kumar Lohia

Suraksha Vidyarthi

Prateek Goyal

Arunkumar Kalpathi Suryanarayanan

Swapna Somineni

Prashant Pandurang Mundhe

Gregory Leonidovich Klets

Padmaja U Mantravadi

A machine learning model data quality improvement detection tool is provided for identifying an accurate reference group and an accurate monitored group of a machine learning model. The tool monitors a behavior of the machine learning model for a predetermined time frame. The tool compares a determined fairness metric a pre-defined fairness threshold. Responsive to the fairness metric failing to meet the pre-defined fairness threshold, the tool modifies the monitored group to include a first portion of the reference group. The tool compares a newly determined fairness metric to the pre-defined fairness threshold. Responsive to the newly determined fairness metric meeting the pre-defined fairness threshold, the tool identifies the modified monitored group including the first portion of the user-defined reference group as a new monitored group and the modified reference group without the first portion of the user-defined reference group as a new reference group.

Decreasing error in a machine learning model based on identifying reference and monitored groups of the machine learning model

A computer-implemented method according to one embodiment includes identifying an occurrence of accuracy drift by a trained model; identifying data associated with the accuracy drift, utilizing a drift detection model (DDM) constructed for the trained model; applying the data associated with the accuracy drift to a decision tree to determine a feature space and specific subset of the data causing the accuracy drift; analyzing a distribution of features within the feature space for the specific subset of the data causing the accuracy drift to determine specific features of the data causing the accuracy drift; and returning the specific features of the data causing the accuracy drift.

Explaining accuracy drift in production data

An example operation may include one or more of storing a batch scoring engine and an application programming interface (API) for the batch scoring engine, receiving a trigger to perform a batch prediction process, reading input data from a source data store and executing, via the batch scoring engine, one or more predictive models on the input data to generate a predictive output and metadata associated with the predictive output, storing the predictive output and the metadata in a target data store, and updating the API with a location of the predictive output within the target data store and a location of the metadata within the target data store.

API management for batch processing

Early indications of application programming interface (API) usage are identified by correlation to particular issues with the API including singular and mutual consistency, completeness, accuracy, and staleness. Analysis of API input and output along with data type and formatting information facilitates identification of the API issues. Establishing a correlation between API usage and issues supports early detection of potential usage reduction on a case-by-case level. Corrective action to resolve identified issues may be performed in a timely manner to maintain usage levels.

Correlating usage to issues in application programming interfaces

Artificial intelligence (AI) model monitoring and ranking includes obtaining metric values indicative of performance of AI model deployments, the metric values including respective metric values measured across metrics, determining violation statuses of the metrics for each of the AI model deployments, the violation statuses indicating, for each AI model deployment, which of the metrics are violated by the AI model deployment as reflected by respective metric values for that AI model deployment, ranking the AI model deployments against each other according to a ranking model and based on the determined violation statuses for each of the AI model deployments, and providing a rank of at least some of the AI model deployments to a user.

Artificial intelligence model monitoring and ranking

A processor may receive an original dataset. The processor may segment, automatically, the original dataset into a plurality of data groups. The plurality of data groups may include a model training dataset and a holdout dataset. The processor may generate a model with the model training dataset. The processor may validate the model with the holdout dataset.

Validation of AI models using holdout sets

A system includes a memory having instructions therein and at least one processor in communication with the memory. The at least one processor is configured to execute the instructions to run a machine learning base model on input data to generate base model prediction data and run a machine learning error prediction model on the input data to generate error prediction data. The at least one processor is configured to execute the instructions to generate predicted correct base model prediction data based on the base model prediction data and the error prediction data. The at least one processor is configured to execute the instructions to generate confusion values data based on the base model prediction data and the predicted correct base model prediction data. The at least one processor is also configured to execute the instructions to generate base model accuracy fairness metrics data based on the confusion values data.

Machine learning model accuracy fairness

A source of bias identification (SoBI) tool is provided that identifies sources of bias in a dataset. A bias detection operation is performed on results of a computer model, based on an input dataset, to generate groupings of values for a protected attribute corresponding to a detected bias in the operation of the computer model. The SoBI tool generates a plurality of sub-groups for each grouping of values. Each sub-group comprises an individual value, or a sub-range, for the protected attribute. The SoBI tool analyzes each of the sub-groups in the plurality of sub-groups, based on at least one source of bias identification criterion, to identify one or more sources of bias in the input dataset. The SoBI tool outputs a bias notification to an authorized computing device specifying the one or more sources of bias in the input dataset.

Bias source identification and de-biasing of a dataset

The method provides for one or more processor receiving on a personal device, a mixture of sounds within a sound stream from multiple sources. The one or more processors identifying one or more sounds of the mixture of sounds from the multiple sources, based on a sound separation technique. The one or more processors displaying on a user interface of the personal device an icon corresponding respectively to a classification of the one or more sounds identified from the multiple sources. The one or more processors receiving a selection of a sound from the mixture of the multiple sounds, based on an action by a user of the personal device selecting the icon displayed on the user interface of the personal device, and the one or more processors recording the sound from the mixture of the multiple sounds selected by the user.

Recording a separated sound from a sound stream mixture on a personal device

Managing notifications is provided. Personal monitoring system inputs corresponding to each member of a defined group performing a common task are contextually analyzed to identify a notification sequence for each respective member enabling task performance in a synchronized manner. Progress of each respective member while performing activities corresponding to the common task is analyzed using the personal monitoring system inputs to enable dynamic modification of the notification sequence and content to the members in accordance with the progress. Existence of any problem is identified during performance of activities corresponding to the common task to accordingly modify the notification sequence and content to target members for mitigation of an existing problem. Alignment of one or more members with a completion timeline for a given activity corresponding to the common task is identified for automatic notification suppression of a planned notification upon completion of the given activity within the completion timeline.

Notification management to a group performing a common task

For seamlessly abstracting metadata in multiple formats, an abstraction module converts first metadata of an incoming object and additional metadata for the incoming object from an annotation map contained by the first metadata into a common format useable. The additional metadata is not part of an original format of the first metadata. A communication module communicates the converted metadata to an adapter in the common format, the adapter processing the first metadata and the additional metadata independent of the original format of the first metadata.

Seamless multiple format metadata abstraction

A computer program product comprises a computer useable medium. The computer useable medium has a computer readable program such that when the computer readable medium is executed on a computer, the computer is caused to receive a request from a database driver that is compatible with a software application. The request is in a database language format. Further, the computer is caused to parse the request. In addition, the computer is caused to convert the request into an intermediary data format. The computer is also caused to provide the request in the intermediary data format to a common client interface builder that reformats the request into a format that is compatible with a system resource adapter and invokes the system resource adapter with the reformatted request.

Method and system for providing access to adapters

An apparatus, system, and method are disclosed for providing seamless multiple format metadata abstraction. A metadata application programming interface (API) is provided to abstract metadata from an incoming business object and provide it to an adapter for processing. The metadata API is configured to operate on multiple metadata formats such as SDO and JavaBean such that the adapter receives the metadata from the metadata API in a common format that is compatible with the adapter. In this manner, the multiple metadata formats may be plugably used with the adapter.

Growing community of inventors

Hyderabad Telangana, India

Madhavi Katari