International Business Machines Corporation

Texas Instruments Corporation

Xiaodong Cui

Wei Zhang

Alper Buyuktosunoglu

Abdullah Kayi

Vaibhava Goel

Brian E D Kingsbury

Xin Zhang

Yada Zhu

Jin Ping Han

Dennis Merton Newns

George Andrei Saon

Youssef Mroueh

Guangnan Ye

Mingrui Liu

Li Feng Zhang

Luis Alfonso Lastras-Montano

Paul Michael Solomon

Rui Zhang

Siyuan Lu

Ulrich Alfons Finkler

David Shing-Ki Kung

Kumar Bhaskaran

Conrad M Albrecht

Marcus Oliver Freitag

David C Haws

Kaisheng N Yao

Upendra D Chitnis

Hui Wan

Zoltan Tueske

Di Chen

One or more systems, devices, computer program products and/or computer-implemented methods of use provided herein relate to length perturbation techniques for improving generalization of DNN acoustic models. A computer-implemented system can comprise a memory that can store computer executable components. The computer-implemented system can further comprise a processor that can execute the computer executable components stored in the memory, wherein the computer executable components can comprise a frame skipping component that can remove one or more frames from an acoustic utterance via frame skipping. The computer executable components can further comprise a frame insertion component that can insert one or more replacement frames into the acoustic utterance via frame insertion to replace the one or more frames with the one or more replacement frames to enable length perturbation of the acoustic utterance.

Length perturbation techniques for improving generalization of deep neural network acoustic models

Hessian matrix-free sample-based techniques for model explanations that are faithful to the model are provided. In one aspect, a method for explaining a machine learning model {circumflex over (θ)} (e.g., for natural language processing) is provided. The method includes: training the machine learning model {circumflex over (θ)} with training data D; obtaining a decision of the machine learning model {circumflex over (θ)}; and explaining the decision of the machine learning model {circumflex over (θ)} using training examples from the training data D.

Faithful and efficient sample-based model explanations

A machine learning model can be trained to predict one or more financial indicators using earnings call transcripts augmented with counterfactual information. Using faithful gradient-based method, prediction results with respect to a particular counterfactual information can be explained. Based on the explanation, the counterfactual information determined to have most impact on prediction results can be selected for updating the machine learning model.

Counterfactual neural network learning for contextual enhanced earnings call analysis

Mechanisms are provided for performing machine learning training of a computer model. A perturbation generator generates a modified training data comprising perturbations injected into original training data, where the perturbations cause a data corruption of the original training data. The modified training data is input into a prediction network of the computer model and processing the modified training data through the prediction network to generate a prediction output. Machine learning training is executed of the prediction network based on the prediction output and the original training data to generate a trained prediction network of a trained computer model. The trained computer model is deployed to an artificial intelligence computing system for performance of an inference operation.

Reducing exposure bias in machine learning training of sequence-to-sequence transducers

Embodiments of a method are disclosed. The method includes performing distributed deep learning training on multiple batches of training data using corresponding learners. Additionally, the method includes determining training times wherein the learners perform the distributed deep learning training on the batches of training data. The method also includes modifying a processing aspect of the straggler to reduce a future training time of the straggler for performing the distributed deep learning training on a new batch of training data in response to identifying a straggler of the learners by a centralized control.

Dynamic computation rates for distributed deep learning

A method for power saving and encryption during analysis of media captured by an information capture device using a partitioned neural network includes replicating, by an information capture device, an artificial neural network (ANN) from a computer server to the information capture device. The ANN on the computer server and a replicated ANN, both, include M layers. The method further includes, in response to captured data being input to be processed, partially processing, by the information capture device, the captured data by executing a first k layers using the replicated ANN, wherein only the k layers are selected to execute on the information capture device. The method further includes transmitting, by the information capture device, an output of the k-th layer to the computer server, which partially processes the captured data by executing the remainder of the M layers using the ANN and the output of the k-th layer.

Media capture device with power saving and encryption features for partitioned neural network

From metadata of a corpus of natural language text documents, a relativity matrix is constructed, a row-column intersection in the relativity matrix corresponding to a relationship between two instances of a type of metadata. An encoder model is trained, generating a trained encoder model, to compute an embedding corresponding to a token of a natural language text document within the corpus and the relativity matrix, the encoder model comprising a first encoder layer, the first encoder layer comprising a token embedding portion, a relativity embedding portion, a token self-attention portion, a metadata self-attention portion, and a fusion portion, the training comprising adjusting a set of parameters of the encoder model.

Transformer-based encoding incorporating metadata

Embodiments of a method are disclosed. The method includes performing distributed deep learning training on a batch of training data. The method also includes determining training times representing an amount of time between a beginning batch time and an end batch time. Further, the method includes modifying a communication aspect of the communication straggler to reduce a future network communication time for the communication straggler to send a future result of the distributed deep learning training on a new batch of training data in response to the centralized parameter server determining that the learner is the communication straggler.

Dynamic network bandwidth in distributed deep learning training

Embodiments of a method are disclosed. The method includes performing decentralized distributed deep learning training on a batch of training data. Additionally, the method includes determining a training time wherein the learner performs the decentralized distributed deep learning training on the batch of training data. Further, the method includes generating a table having the training time and other processing times for corresponding other learners performing the decentralized distributed deep learning training on corresponding other batches of other training data. The method also includes determining that the learner is a straggler based on the table and a threshold for the training time. Additionally, the method includes modifying a processing aspect of the straggler to reduce a future training time of the straggler for performing the decentralized distributed deep learning training on a new batch of training data in response to determining the learner is the straggler.

Dynamic computation in decentralized distributed deep learning training

Systems, computer-implemented methods, and computer program products to facilitate updating, such as averaging and/or training, of one or more statistical sets are provided. According to an embodiment, a system can comprise a memory that stores computer executable components and a processor that executes the computer executable components stored in the memory. The computer executable components can include a computing component that averages a statistical set, provided by the system, with an additional statistical set, that is compatible with the statistical set, to compute an averaged statistical set, where the additional statistical set is obtained from a selected additional system of a plurality of additional systems. The computer executable components also can include a selecting component that selects the selected additional system according to a randomization pattern.

Updating of statistical sets for decentralized distributed training of a machine learning model

Building machine learning models by receiving, a plurality of training process scores associated with the model parameter lists, determining, a best model parameter list according to the training process scores, determining a descendant model parameter list according to the best model parameter list, wherein the descendant parameter list comprises a portion of the best model parameter list, distributing the descendant model parameter list, conducting a model training process according to the descendant model parameter list, determining a training process score according to the descendant model parameter list, and sending the training process score for the descendant model parameter list.

Asynchronous multiple scheme meta learning

Embodiments of a method are disclosed. The method includes performing a batch of decentralized deep learning training for a machine learning model in coordination with multiple local homogenous learners on a deep learning training compute node, and in coordination with multiple super learners on corresponding deep learning training compute nodes. The method also includes exchanging communications with the super learners in accordance with an asynchronous decentralized parallel stochastic gradient descent (ADPSGD) protocol. The communications are associated with the batch of deep learning training.

Hierarchical decentralized distributed deep learning training

A task-based learning using task-directed prediction network can be provided. Training data can be received. Contextual information associated with a task-based criterion can be received. A machine learning model can be trained using the training data. A loss function computed during training of the machine learning model integrates the task-based criterion, and minimizing the loss function during training iterations includes minimizing the task-based criterion.

Task-oriented machine learning and a configurable tool thereof on a computing environment

Techniques for image processing and transformation are provided. A plurality of images and a plurality of maps are received, and a system of neural networks is trained based on the plurality of images and the plurality of maps. A first image is received, and a first map is generated by processing the first image using the system of neural networks.

Growing community of inventors

Chappaqua, NY, United States of America

Xiaodong Cui