International Business Machines Corporation

Other

Parijat Dube

Li Feng Zhang

Hong Min

Sameh W Asaad

Bharat Sukhwani

Mathew S Thoennes

Zhen Liu

Xiaoqiao Meng

Eric P Bouillet

Dimitrios Pendarakis

Laura Wynter

Seetharami R Seelam

Xavier Rene Guerin

Kay Schwendimann Anderson

Bishwaranjan Bhattacharjee

Andrzej P Kochut

Alexei A Karve

Joseph Phillip Bigus

Anshul Gandhi

Balakrishna Raghavendra Iyer

Michael H Dawson

Zehra Noman Sura

Rahul Jain

Michel H T Hack

Jian Tan

Graeme Johnson

David Alson George

Patrick D Watson

John Ronald Kender

Priya Ashok Nagpurkar

Gauri Joshi

Michael J Branson

Donald W Schmidt

Wei Tan

Frederick Douglis

John Alan Bivens

Zachary Adam Garbow

Liana Liyow Fong

Milind R Naphade

Yuqing Gao

Raymond Byars Jennings, Iii

Michael Robert Glass

Matthew Leon Hill

Noel C Codella

Megumi Ito

Donna N Eng Dillenberger

Bernard V Brezzo

Diana Jeanne Arroyo

Michael M Tsao

Yanbin Liu

Siyu Huo

Stefania Costache

Pramod B Nagaraja

Brian Michael Belgodere

Nai K Ling

Giuseppe A Paleologo

Sambit Sahu

Eitan D Farchi

Ashish Verma

Tong Chen

Nagui Halim

Vinod Muthusamy

Mark David Feblowitz

Karthik Subbian

Kaoutar El Maghraoui

Ramakrishnan Kannan

Honghui Xia

George V Popescu

Malgorzata Elzbieta Stys

Joao P Goncalves

Matthew S Thoennes

Falk Pollok

Liana L Fo

Rahul Jan

Sanghamitra Dutta

Sidharth Gupta

In an embodiment, a method for fine-tuning a pre-trained neural network for transfer learning, the method comprising obtaining a first target feature vector from a first layer of a pre-trained neural network responsive to a first target data element of a target dataset passing therethrough, obtaining a first source feature vector associated with the first layer of the pre-trained neural network, calculating a first divergence value for the first layer of the pre-trained neural network based at least in part on the first target feature vector and the first source feature vector, and setting a learning rate for the first layer of the pre-trained neural network based at least in part on the first divergence value.

Automated fine-tuning of a pre-trained neural network for transfer learning

Whether to train a new neural network model can be determined based on similarity estimates between a sample data set and a plurality of source data sets associated with a plurality of prior-trained neural network models. A cluster among the plurality of prior-trained neural network models can be determined. A set of training data based on the cluster can be determined. The new neural network model can be trained based on the set of training data.

Training transfer-focused models for deep learning

Techniques for estimating runtimes of one or more machine learning tasks are provided. For example, one or more embodiments described herein can regard a system that can comprise a memory that stores computer executable components. The system can also comprise a processor, operably coupled to the memory, and that can execute the computer executable components stored in the memory. The computer executable components can comprise an extraction component that can extract a parameter from a machine learning task. The parameter can define a performance characteristic of the machine learning task. Also, the computer executable components can comprise a model component that can generate a model based on the parameter. Further, the computer executable components can comprise an estimation component that can generate an estimated runtime of the machine learning task based on the model. The estimated runtime can define a period of time beginning at an initiation of the machine learning task and ending at a completion of the machine learning task.

Runtime estimation for machine learning tasks

A neural network models fragmenting method, system, and computer program product include recursively factoring out common prefixes of models, constructing a hierarchy of decomposed model fragments based on the factoring, and grouping the constructed hierarchy for deployment.

Scalable model serving

Embodiments for implementing mixed precision learning for neural networks by a processor. A neural network may be replicated into a plurality of replicated instances and each of the plurality of replicated instances differ in precision used for representing and determining parameters of the neural network. Data instances may be routed to one or more of the plurality of replicated instances for processing according to a data pre-processing operation.

Data driven mixed precision learning for neural networks

Systems, computer-implemented methods, and computer program products that can facilitate switching a model training process from a ground truth training phase to an adversarial training phase based on performance of a model trained in the ground truth training phase are provided. According to an embodiment, a system can comprise a memory that stores computer executable components and a processor that executes the computer executable components stored in the memory. The computer executable components can comprise an analysis component that identifies a performance condition of a model trained in a model training process. The computer executable components can further comprise a trainer component that switches the model training process from a ground truth training process to an adversarial training process based on the identified performance condition.

Performance based switching of a model training process

A method for performing machine learning includes assigning processing jobs to a plurality of model learners, using a central parameter server. The processing jobs includes solving gradients based on a current set of parameters. As the results from the processing job are returned, the set of parameters is iterated. A degree of staleness of the solving of the second gradient is determined based on a difference between the set of parameters when the jobs are assigned and the set of parameters when the jobs are returned. The learning rates used to iterate the parameters based on the solved gradients are proportional to the determined degrees of staleness.

Adaptive learning rate schedule in distributed stochastic gradient descent

A computer-implemented method for data labeling is provided. The computer-implemented method assigns pseudo-labels to unlabeled examples of data using a similarity metric on an embedding space to produce pseudo-labeled examples. A curriculum learning model is trained using the pseudo-labeled examples. The curriculum learning model trained with the pseudo-labeled examples is employed in in a fine-tuning task to enhance classification accuracy of the data.

Generating and augmenting transfer learning datasets with pseudo-labeled images

A method and system of simulating a cost of shipment are provided. A request for quote (RFQ) is received by a computing device from a shipper having a plurality of trade lanes. A revenue generated from each trade lane is estimated and ranked based on the estimated revenue. An original time limit is assigned to each trade lane. A trade lane with a highest ranking that has not yet been selected is selected. For the selected trade lane, graphs for space and time granularity analysis are generated. Space and time granularities that maximize accuracy within the assigned time limit based on the generated graphs are calculated. Cost simulation analysis is performed using the calculated space and time granularities. Upon determining that there are trade lanes not yet selected, there is a return to the act of selecting a trade lane.

Adjusting simulation times for cost simulation analysis of transportation lane proposals based on space and time granularities

Embodiments include methods, systems and computer program products for offloading multiple processing operations to an accelerator. Aspects include receiving a database query from an application, performing an analysis on the query, and identifying a plurality of available accelerators. Aspects further include retrieving cost information for one or more templates available on each of the plurality of available accelerators, determining a query execution plan based on the cost information and the analysis on the query, and offloading one or more query operations to at least one of the plurality of accelerators based on the query execution plan.

Scalable acceleration of database query operations

Embodiments include methods, systems and computer program products a for offloading multiple processing operations to an accelerator includes receiving, by a processing device, a database query from an application. The method also includes performing analysis on the database query and selecting an accelerator template from a plurality of accelerator templates based on the analysis of the database query. The method further includes transmitting an indication of the accelerator template to the accelerator and executing at least a portion of the database query on the accelerator.

Accelerating multiple query processing operations

Methods and arrangements for mapping a query operation to an accelerator are provided. The method includes receiving, by a processor, a query operation and determining the design logic of the query operation, receiving a configuration of one or more available accelerators and a design logic of each of the one or more available accelerators, and determining if the query operation can be offloaded to one or more of the one or more available accelerators. Based on a determination that the query operation can be offloaded to one or more of the one or more available accelerators, the method also includes creating software structures to interface with a selected accelerator and executing the query operation on the selected accelerator. Based on a determination that the query operation cannot be offloaded to one or more of the one or more available accelerators, the method further includes executing the query operation in software.

Mapping query operations in database systems to hardware based query accelerators

A computer-implemented method includes determining that a database query warrants a first projection operation to project a plurality of input rows to a plurality of projected rows, where each of the plurality of input rows has one or more variable-length columns. A first projection control block is constructed, by a computer processor, to describe the first projection operation. The first projection operation is offloaded to a hardware accelerator. The first projection control block is provided to the hardware accelerator, and the first projection control block enables the hardware accelerator to perform the first projection operation at streaming rate.

Offloading projection of fixed and variable length database columns

A data processing system includes a plurality of virtual machines each having associated memory pages; a shared memory page cache that is accessible by each of the plurality of virtual machines; and a global hash map that is accessible by each of the plurality of virtual machines. The data processing system is configured such that, for a particular memory page stored in the shared memory page cache that is associated with two or more of the plurality of virtual machines, there is a single key stored in the global hash map that identifies at least a storage location in the shared memory page cache of the particular memory page. The system can be embodied at least partially in a cloud computing system.

Method, apparatus and computer programs providing cluster-wide page management

Growing community of inventors

Yorktown Heights, NY, United States of America

Parijat Dube