Servicenow Canada Inc.

Jan Freyberg

Lorne Schell

Frédéric Branchaud-Charron

Parmida Atighehchian

A method and system for managing a dataset. An artificial intelligence (AI) model is to be used on the dataset. A data mask describes a labeling status of the data items. A loop is repeated until patience parameters are satisfied. The loop comprises receiving trusted labels provided by trusted labelers; updating the data mask; from a labelled data items subset, training the AI model; cloning the trained AI model into a local AI model on processing nodes; creating and chunking a randomized unlabeled subset into data subsets for dispatching to the processing nodes; receiving an indication that predicted label answers have been inferred by the processing nodes using the local AI model; computing a model uncertainty measurement from statistical analysis of the predicted label answers. The patience parameters include one or more of a threshold value on the model uncertainty measurement and information gain between different training cycles.

Method and server for managing a dataset in the context of artificial intelligence

A method and server for optimizing hyperparameter tuples for training production-grade artificial intelligence (AI) models. For each one of the AI models, AI model features are extracted and, for the one AI model, an initial distribution of n hyperparameter tuplesis created considering the extracted AI model features therefor. A loop is repeated, until metric parameters are satisfied, comprising: evaluating latency from training the one AI model for each of the n hyperparameters tuples; evaluating model uncertainty from training the one AI model for each of the n hyperparameters tuples; for each of the n hyperparameters tuples, computing a blended quality measurement from the evaluated latency and evaluated model uncertainty; replacing m hyperparameter tuples having the worst blended quality measurements with m newly generated hyperparameter tuples. The metric parameters include one or more of a threshold value on model uncertainty and blended quality measurement gain between successive loops.

Growing community of inventors

Montreal, Canada

Jan Freyberg