International Business Machines Corporation

Lam Nguyen

Dzung Tien Phan

Pavankumar Murali

Thanh Lam Hoang

Dung Tien Phan

Gabriele Picco

Jayant R Kalagnanam

Nianjun Zhou

Vanessa Lopez Garcia

Hongsheng Liu

Roman Vaculin

Marco Luca Sbodio

Yufang Hou

Nam H Nguyen

Stephane Michel

Yuan Yuan Jia

Surya Shravan Kumar Sajja

Kanthi Sarpatwar

Haoran Zhu

Ramon Fernandez Astudillo

Young-Suk Lee

Binny Winston Samuel

An approach to federated learning of a machine learning model may be provided. The approach may include broadcasting hyperparameters of a machine learning model to one or more client computing devices from a primary device associated with an outer loop or an inner loop. A gradient for the loss function may be calculated at the client device if previous gradients have been sufficiently large. If gradients exceeds a threshold, the client can send the mini-batch of gradients or the difference of the mini-batch of gradients back to the primary device. A search direction may be calculated based on the full gradient of the loss function for an outer loop or the mini-batch of gradient differences for an inner loop. A learning rate step may be calculated from the search direction. The hyperparameter may be updated for the inner loop based on the learning rate.

Federated learning for training machine learning models

Aspects of the invention include training an optimal interpretable decision tree for regression using mixed-integer linear programming techniques. A non-limiting example computer-implemented method includes receiving, using a processor, input data that includes time-series data. The method further includes training, using a binary mixed-integer linear program of the processor, an ODT for regression based on the input data. During the training process one or more outliers are filtered out by a linear loss model that minimizes training loss and outlier loss.

System-level control using tree-based regression with outlier removal

A computer implemented method for automatically generating an optimization model for site-wide plant optimization includes mapping a process flow diagram of a plant process to a graph comprising nodes and edges, wherein the nodes represent processes and the edges represent flows between processes. A behavior is learned for each node of the graph based at least on historic data of the plant process. One or more regression functions are modeled for each node to predict an output of each of the processes, wherein the one or more regression functions are modeled based on the learned behavior for each node.

Automated generation of optimization model for system-wide plant optimization

A computer implemented method using an artificial intelligence (A.I.) module to explain large scale scheduling solutions includes receiving an original instance of a resource constrained scheduling problem. The instance includes a set of tasks and a variety of resource requirements and a variety of constraints. An optimizer process determines a schedule for the set of tasks while minimizing a makespan of the schedule. A minimal set of resource links is generated based on resource dependencies between tasks. The resource links are added to the original instance of scheduling problem, as precedence constraints. All the resource constraints are removed from the original instance of the resource constrained scheduling problem. A set of critical tasks is computed using a non-resource constrained critical path. Schedules are provided with an explanation of an optimized order of the set of tasks based on the use of the non-resource constrained critical path.

Optimizer agnostic explanation system for large scale schedules

A method, computer program product and system are provided for feature engineering and synthetic data generation. A processor retrieves a plurality of data tables, where the plurality of data tables are heterogeneous in format and content. A processor trains a variational auto-encoder (VAE) model on the plurality of data tables. A processor receives an input data table. A processor generates a synthetic data table based on the input data table and the trained VAE model.

Unsupervised learning from public tabular datasets

Dynamic control of a production process of a manufacturing system is facilitated, where the control process includes receiving runtime input data for multiple input variables of the production process. The production process is represented, at least in part, by a physics-based expression, with at least one term of the physics-based expression being a function of two or more input variables of the production process. The control process includes determining coefficient and bias terms for a dynamic linear model connecting the multiple input variables and an output of the production process, where the terms are based, at least in part, on the input variables. The dynamic linear model and determined coefficient and bias terms are provided in an optimization model to generate a regression-optimization model which determines an optimized value of a control variable for the production process, which is used in facilitating control of the production process.

Regression-optimization control of production process with dynamic inputs

Software that performs the following operations: (i) receiving a set of graph predictions corresponding to an input text, where graph predictions of the set of graph predictions are generated by different respective machine learning models; (ii) blending the graph predictions of the set of graph predictions to generate a plurality of candidate blended graphs, where nodes and edges of the candidate blended graphs have respective selection metric values, generated using a selection metric function, that meet a minimum threshold; and (iii) selecting as an output blended graph a candidate blended graph of the plurality of candidate blended graphs having a highest total combination of selection metric values among the plurality of candidate blended graphs.

Blending graph predictions

Embodiments are provided for unsupervised learning of domain specific knowledge graph from textual data and language generation from knowledge graph via reinforcement learning in a computing system by a processor. Unstructured data is automatically parsed into one or more knowledge graphs based on the unstructured data and a list of candidate relations using a first machine learning model. Text data is generated from the one or more knowledge graphs using a second machine learning model.

Providing a semantic encoding and language neural network

Embodiments are provided for generating a reasonable language model learning for text data in a knowledge graph in a computing system by a processor. One or more data sources and one or more triples may be analyzed from a knowledge graph. Training data having one or more candidate labels associated with one or more of the triples may be generated. One or more reasonable language models may be trained based on the training data.

Reasonable language model learning for text generation from a knowledge graph

In example aspects of this disclosure, a method includes generating, by one or more computing devices, a parametric model that expresses condition states for each of a plurality of assets, and the probability of the assets transitioning between the condition states; generating, by the one or more computing devices, stochastic degradation predictions of a group of the assets, based on the condition states and the probability of transitioning between the condition states for at least some of the assets; and generating, by the one or more computing devices, a maintenance schedule based on: the stochastic degradation predictions of the group of the assets, costs of corrective maintenance for assets in a failed state, and costs of scheduled maintenance for the assets.

Intelligent dynamic condition-based infrastructure maintenance scheduling

In an approach for compressing a neural network, a processor receives a neural network, wherein the neural network has been trained on a set of training data. A processor receives a compression ratio. A processor compresses the neural network based on the compression ratio using an optimization model to solve for sparse weights. A processor re-trains the compressed neural network with the sparse weights. A processor outputs the re-trained neural network.

Compression of deep neural networks

In an approach to hyperparameter optimization, one or more computer processors express a hyperparameter tuning process of a model based on a type of model, one or more dimensions of a training dataset, associated loss function of the model, and associated computational constraints of the model, comprising: identifying a set of optimal hyper-rectangles based a calculated local variability and a calculated best function value; calculating a point as a representative for each identified potentially optimal hyper-rectangle by locally searching over the identified set of potentially optimal hyper-rectangles; dividing one or more hyper-rectangles in the identified set of optimal hyper-rectangles into a plurality of smaller hyper-rectangles based on each calculated point; and calculating one or more optimal hyperparameters utilizing a globally converged hyper-rectangle from the plurality of smaller hyper-rectangles. The one or more computer processors classify one or more unknown datapoints utilizing the model associated with tuned hyperparameters.

Tuning classification hyperparameters

Aspects of the invention include an optimal interpretable decision tree using integer linear programming techniques. A non-limiting example computer-implemented method includes receiving, using a processor, a plurality of data inputs from a process and selecting, using the processor, a data subset from the plurality of data inputs by solving linear programming to obtain a solution. The method builds and optimizes, using the processor, an optimal decision tree based on the data subset and alerts, using the processor, a user when a prediction of the optimal decision tree is greater than a threshold value.

Optimal interpretable decision trees using integer linear programming techniques

Aspects of the invention include implemented method includes selecting an optimization algorithm for the control system of a processing plant based on whether the control system is guided by a linear-based predictive model or a non-linear-based predictive model, in which a gradient is available. Calculating set-point variables using the optimization algorithm. Predicting an output based on the calculated set-point variables. Comparing an actual output at the processing plant to the predicted output. Suspending a physical process at the processing plant in response to the actual output being a threshold value apart from the predicted output.

Site-wide operations management optimization for manufacturing and processing control

A computer-implemented method for a shuffling-type gradient for training a machine learning model using a stochastic gradient descent (SGD) includes the operations of uniformly randomly distributing data samples or coordinate updates of a training data, and calculating the learning rates for a no-shuffling scheme and a shuffling scheme. A combined operation of the no-shuffling scheme and the shuffling scheme of the training data is performed using a stochastic gradient descent (SGD) algorithm. The combined operation is switched to performing only the shuffling scheme from the no-shuffling scheme based on one or more predetermined criterion; and training the machine learning models with the training data based on the combined no-shuffling scheme and shuffling scheme.

Growing community of inventors

Ossining, NY, United States of America

Lam Nguyen