International Business Machines Corporation

Arunkumar Kalpathi Suryanarayanan

Manish Anand Bhide

Ravi Chandra Chamarthy

Madhavi Katari

Harivansh Kumar

Prashant Pandurang Mundhe

Swapna Somineni

A machine learning model data quality improvement detection tool is provided for identifying an accurate reference group and an accurate monitored group of a machine learning model. The tool monitors a behavior of the machine learning model for a predetermined time frame. The tool compares a determined fairness metric a pre-defined fairness threshold. Responsive to the fairness metric failing to meet the pre-defined fairness threshold, the tool modifies the monitored group to include a first portion of the reference group. The tool compares a newly determined fairness metric to the pre-defined fairness threshold. Responsive to the newly determined fairness metric meeting the pre-defined fairness threshold, the tool identifies the modified monitored group including the first portion of the user-defined reference group as a new monitored group and the modified reference group without the first portion of the user-defined reference group as a new reference group.

Decreasing error in a machine learning model based on identifying reference and monitored groups of the machine learning model

A method, system, and computer program product for generating lineage events of machine learning models. The method may include identifying a machine learning model with missing lineage. The method may also include generating a creation event and deployment event for the machine learning model. The method may also include generating a version change event for the machine learning model. Generating the version change event may include identifying one or more predicted data points with a low model confidence; rescoring the one or more predicted data points based on the machine learning model at a second time period; determining that the updated one or more predicted data points are significantly different than the one or more predicted data points; and determining that there is a new version of the machine learning model. The method may also include creating a lineage path for the machine learning model.

Generation of machine learning model lineage events

One example of a system comprises using a processor for identifying a model to be validated that is stored in a repository; automatically computing and recording one or more model metrics for the model to be validated in a tamper-proof manner; comparing the computed tamper-proof metrics with one or more encoded rules and policies to determine if the model to be validated complies with the one or more encoded rules and policies; and outputting a notification to a device indicating a validation status of the model to be validated based on the comparison of the computed tamper-proof metrics with the one or more encoded rules and policies.

AI governance using tamper proof model metrics

Artificial intelligence (AI) model monitoring and ranking includes obtaining metric values indicative of performance of AI model deployments, the metric values including respective metric values measured across metrics, determining violation statuses of the metrics for each of the AI model deployments, the violation statuses indicating, for each AI model deployment, which of the metrics are violated by the AI model deployment as reflected by respective metric values for that AI model deployment, ranking the AI model deployments against each other according to a ranking model and based on the determined violation statuses for each of the AI model deployments, and providing a rank of at least some of the AI model deployments to a user.

Artificial intelligence model monitoring and ranking

One example of a method comprises identifying a model to be validated that is stored in a repository; automatically computing and recording one or more model metrics for the model to be validated in a tamper-proof manner; comparing the computed tamper-proof metrics with one or more encoded rules and policies to determine if the model to be validated complies with the one or more encoded rules and policies; and outputting a notification to a device indicating a validation status of the model to be validated based on the comparison of the computed tamper-proof metrics with the one or more encoded rules and policies.

Growing community of inventors

Chennai, India

Arunkumar Kalpathi Suryanarayanan