International Business Machines Corporation

Saurabh Manish Raje

Yogish Sabharwal

Ashish Verma

Vaibhav Saxena

Parikshit Ram

Aswin Kannan

Anamitra Roy Choudhury

Venkatesan T Chakaravarthy

Saurabh Goyal

A system, computer program product, and method are presented for performing multi-objective automated machine learning, and, more specifically, to identifying a plurality of machine learning pipelines as Pareto-optimal solutions to optimize a plurality of objectives. The method includes receiving input data directed toward one or more subjects of interest and determining a plurality of objectives to be optimized. The method also includes ingesting at least a portion of the input data through one or more machine learning (ML) models. The method further includes aggregating the plurality of objectives into one or more aggregated single objectives. The method also includes determining a plurality of Pareto-optimal solutions, thereby defining a plurality of ML pipelines that optimize the one or more aggregated single objectives. The method further includes selecting one ML pipeline from the plurality of ML pipelines.

Multi-objective automated machine learning

Methods, systems, and computer program products for accelerating inference of transformer-based models are provided herein. A computer-implemented method includes obtaining a machine learning model comprising a plurality of transformer blocks, a task, and a natural language dataset; generating a compressed version of the machine learning model based on the task and the natural language dataset, wherein the generating comprises: obtaining at least one set of tokens, wherein each token in the set corresponds to one of the items in the natural language dataset, identifying and removing one or more redundant output activations of different ones of the plurality of transformer blocks for the at least one set of tokens, and adding one or more input activations corresponding to the one or more removed output activations into the machine learning model at subsequent ones of the plurality of the transformer blocks; and outputting the compressed version of the machine learning model to at least one user.

Accelerating inference of transformer-based models

Growing community of inventors

Gurgaon, India

Saurabh Manish Raje