Intuit, Inc.

Sumanth Venkatasubbaiah

Caio Vinicius Soares

Qingbo Hu

Abhishek Jain

Andreas Mavrommatis

Pankaj Rastogi

Karthik Prakash

Nicholas Jeffrey Hoh

Yuwen Ellen Wu

Ashok N Srivastava

Sricharan Kallur Palli Kumar

Frank Wisniewski

Efraim David Feinstein

Manas Kumar Mukherjee

Malhar Siddhesh Jere

The present disclosure provides techniques for schema-based machine learning model monitoring. One example method includes receiving input data to and output data related to a machine learning model, wherein the input data and the output data conform to a data schema, retrieving, based on the data schema, a set of fields associated with the input data and the output data, performing statistical analysis for the machine learning model based on the set of fields retrieved, and predicting one or more attributes of the machine learning model based on the statistical analysis, wherein the one or more attributes of the machine learning model indicate a result of monitoring of the machine learning model, explainability information related to the machine learning model, or health of the machine learning model.

Schema-based machine learning model monitoring

Certain aspects of the present disclosure provide techniques for operation of a feature management platform. A feature management platform is an end-to-end platform developed to manage the full lifecycle of data features. For example, to create a stateful feature, the feature management platform can receive a processing artifact from a computing device. The processing artifact defines the stateful feature, including the data source to retrieve event data from, when to retrieve the event data, the type of transform to apply, etc. Based on the processing artifact, the feature management system generates a processing job (e.g., the API defines a pipeline), which when initiated generates a vector that encapsulates the stateful feature. The vector is transmitted to the computing device that locally hosts a model, which generates a prediction that is transmitted to the feature management platform. Subsequently, the predication and stateful feature can be transmitted to other computing devices.

Real time fault tolerant stateful featurization

A method for generating a synthetic dataset involves generating discretized synthetic data based on driving a model of a cumulative distribution function (CDF) with random numbers. The CDF is based on a source dataset. The method further includes generating the synthetic dataset from the discretized synthetic data by selecting, for inclusion into the synthetic dataset, values from a multitude of entries of the source dataset, based on the discretized synthetic data, and providing the synthetic dataset to a downstream application that is configured to operate on the source dataset.

Method and system for generating synthetic data using a regression model while preserving statistical properties of underlying data

A method including extracting, from an input, supported data. The input includes outputs from machine learning models in different formats. The supported data includes a subset of the input after data normalization. The method also includes inferring, from the supported data, data types to be used with respect to generating metrics for the machine learning models. The method also includes generating, from the supported data and using the data types, a relational event including the supported data. The relational event further includes a first data structure object including the types and having a first data structure different than the different formats. The method also includes calculating, using the supported data in the first data structure, the metrics for the machine learning models. The method also includes generating, from the relational event, a monitoring event. The monitoring event includes a second data structure object segmented into data buckets storing the metrics.

Framework for processing machine learning model metrics

Growing community of inventors

Mountain View, CA, United States of America

Sumanth Venkatasubbaiah