International Business Machines Corporation

Berthold Reinwald

Shirish Tatikonda

Matthias Boehm

Yuanyuan Tian

Douglas Ronald Burdick

Peter Haas

Prithviraj Sen

John Sismanis

Alexandre Valentinovich Evfimievski

Mir Hamid Pirahesh

Shivakumar Vaithyanathan

Andrey Balmin

Heasoo Hwang

Sameep Mehta

Ling Wang

Chandrasekaran Mohan

Charles Daniel Wolfson

Saileshwar Krishnamurthy

James C Kleewein

Peter Martin Schwarz

Ariel Fuxman

Larry Justin Brown

Hima Patel

Balaji Ganesan

John D Keenleyside

Sebastian Schelter

Keith W Campbell

Erik Nijkamp

Arash Ashari

Yannis Sismanis

Mehmet Altinel

Bo Gao

Wei Sun

Zhi Hu Wang

Guy Maring Lohman

Petr Novotny

Ning Wang

Li Fan

Soma Shekar Naganna

Mu Qiao

Qi Rong Wang

Kalapriya Kannan

Kevin Scott Beyer

Christof Bornhoevd

Lingfei Wu

Tao Yu

Stefan Bengt Edlund

Chang Jie Guo

Sreeram Viswanath Balakrishnan

Jie Qiu

Rema Ananthanarayanan

Rajasekar Krishnamurthy

Edwin Peter Dawson Pednault

Neeraj Ramkrishna Singh

Tanya Couch

Vikas Sindhwani

Wen Hao An

George Lapis

Shaikh Shahriar Quader

Yuya Jeremy Ong

Shettigar Parkala Srinivas

Rainer Gemulla

Rong Qu

Amol N Ghoting

Adrian Mahjour

Lei Zhi

Aindrila Basak

Rajesh Manjrekar

Riddhiman Dasgupta

Akshay Parekh

Nasrullah Sheikh

Mathias Peters

Xiao Qin

Dheeraj Arremsetty

Carlo Appugliese

Shivam Srivastava

Stefan Burnicki

Akanksha Baid

Alkis Simitsis

Yuen Y Lo

Christof Bomhoevd

Wensheng Wu

Peter Jay Haas

Ling c/o Ibm Uk Ltd Wang

A graph neural network (GNN) training method, system, and computer program product in a graph, include generating, by the computing device, one or more one or more hypothetical edges between two or more nodes of a plurality of nodes of a graph neural network, testing, by the computing device, to determine whether the one or more generated hypothetical edges should be connected by using negative sampling, and permanently connecting, by the computing device, the one or more tested hypothetical edges if the negative sampling indicates the connectivity.

Adaptive self-adversarial negative sampling for graph neural network training

Providing a representative dataset from an initial dataset by accessing a dataset associated with a machine learning model, receiving input parameters associated with the representative dataset selection, the input parameters including an evaluation metric, determining a density of a plurality of datapoints associated with the dataset, training a first iteration of a machine learning model using a first data point selected according to the density, determining a first value of the evaluation metric for the first iteration of the machine learning model, generating a representative subset based on the first value of the evaluation metric value, and providing the representative dataset and a final machine learning model trained using the representative dataset.

Selecting a high coverage dataset

One embodiment provides a computer implemented method, including: obtaining an information document corresponding to an entity, wherein the information document includes redacted information spans; identifying an entity type for each of the redacted information spans, wherein the entity type identifies a relationship between a redacted information span and at least one other entity within the information document; replacing the redacted information spans with replacement entities corresponding to the entity type of a given redacted information span, wherein the replacing is performed in view of a frequency distribution of actual information and wherein the replacing includes maintaining relationships of the redacted information spans; and controlling bias within the replacement entities, wherein the controlling includes detecting bias within the replacement entities.

Generation of training data from redacted information

In an approach for forecasting in multivariate irregularly sampled time series, a processor receives time series data having one or more missing values. A processor determines, from the time series data, non-missing values present in the time series data. A processor determines, from the time series data, zero or more mask values for the time series data. A processor determines time interval values. A processor inputs the one or more missing values, the non-missing values, the zero or more mask values, and the time interval values into a recurrent neural network. A processor determines a predicted value for the one or more missing values.

Forecasting in multivariate irregularly sampled time series with missing values

A computer-implemented method is provided for predicting future data values or target labels of multivariate time series data. The method includes receiving the multivariate time series data having present values, systematic missing values, and random missing values. The method further includes masking the present values, the systematic missing values, and the random missing values using triplet encodings. The method also includes determining time intervals between current missing values, from among the systematic missing values and the random missing values, and immediately preceding ones of the present values. The method additionally includes training, by a computing device, at least one recurrent neural network with the triplet encodings, the time intervals, and multivariate time series data to perform a feedforward pass on the recurrent neural network predicting the future data values or the target labels. A forward imputation is utilized by the computing device to impute any of the current missing values.

Predicting multivariate time series with systematic and random missing values

Aspects of the present disclosure relate to neural-based ontology generation and refinement. A set of input data can be received. A set of entities can be extracted from the set of input data using a named-entity recognition (NER) process, each entity having a corresponding label, the corresponding labels making up a label set. The label set can be compared to concepts in a set of reference ontologies. Labels that match to concepts in the set of reference ontologies can be selected as a candidate concept set. Relations associated with the candidate concepts within the set of reference ontologies can be identified as a candidate relation set. An ontology can then be generated using the candidate concept set and candidate relation set.

Neural-based ontology generation and refinement

A computer-implemented method is provided that includes identifying an input dataset formatted as an input matrix, the input matrix including a plurality of rows and a plurality of columns. The computer-implemented method also includes dividing the input matrix into a plurality of input matrix blocks. Further, the computer-implemented method includes distributing the input matrix blocks to a plurality of different machines across a distributed filesystem, and sampling, by at least two of the different machines in parallel, at least two of the input matrix blocks. Finally, the computer-implemented method includes generating at least one sample matrix based on the sampling of the at least two of the input matrix blocks.

Single-pass distributed sampling from block-partitioned matrices

The embodiments described herein relate to recompiling an execution plan of a machine-learning program during runtime. An execution plan of a machine-learning program is compiled. In response to identifying a directed acyclic graph of high-level operations (HOP DAG) for recompilation during runtime, the execution plan is dynamically recompiled. The dynamic recompilation includes updating statistics and dynamically rewriting one or more operators of the identified HOP DAG, recomputing memory estimates of operators of the rewritten HOP DAG based on the updated statistics and rewritten operators, constructing a directed acyclic graph of low-level operations (LOP DAG) corresponding to the rewritten HOP DAG based in part on the recomputed memory estimates, and generating runtime instructions based on the LOP DAG.

Dynamic recompilation techniques for machine learning programs

A method that includes generating, in a query pre-processor, a set of pre-computed materialized sub-graphs by executing a pre-processing dynamic random-walk based search for a bin of terms. The method also includes receiving, in a query processor, a search query having at least one search query term. In response to receiving the search query, the method includes accessing the set of pre-computed materialized sub-graphs. The accessing includes accessing a text index based on the search query term to retrieve a corresponding term group identifier and accessing the corresponding pre-computed materialized sub-graph based on the term group identifier. The method also includes executing a dynamic random-walk based search on only the corresponding pre-computed materialized sub-graph and based on the executing, retrieving nodes in the dataset and transmitting the nodes as results of the query.

Scaling dynamic authority-based search using materialized subgraphs

A method for global data flow optimization for machine learning (ML) programs. The method includes receiving, by a storage device, an initial plan for an ML program. A processor builds a nested global data flow graph representation using the initial plan. Operator directed acyclic graphs (DAGs) are connected using crossblock operators according to inter-block data dependencies. The initial plan for the ML program is re-written resulting in an optimized plan for the ML program with respect to its global data flow properties. The re-writing includes re-writes of: configuration dataflow properties, operator selection and structural changes.

Global data flow optimization for machine learning programs

Parallel execution of machine learning programs is provided. Program code is received. The program code contains at least one parallel for statement having a plurality of iterations. A parallel execution plan is determined for the program code. According to the parallel execution plan, the plurality of iterations is partitioned into a plurality of tasks. Each task comprises at least one iteration. The iterations of each task are independent. Data required by the plurality of tasks is determined. An access pattern by the plurality of tasks of the data is determined. The data is partitioned based on the access pattern.

Hybrid parallelization strategies for machine learning programs on top of mapreduce

A method for optimization of machine learning (ML) workloads on a graphics processor unit (GPU). The method includes identifying a computation having a generic pattern commonly observed in ML processes. Hierarchical aggregation spanning a memory hierarchy of the GPU for processing is performed for the identified computation including maintaining partial output vector results in shared memory of the GPU. Hierarchical aggregation for vectors is performed including performing intra-block aggregation for multiple thread blocks of a partial output vector results on GPU global memory.

Pipelined approach to fused kernels for optimization of machine learning workloads on graphical processing units

One embodiment provides a method for runtime piggybacking of concurrent data-parallel jobs in task-parallel machine learning (ML) programs including intercepting, by a processor, executable jobs including executable map reduce (MR) jobs and looped jobs in a job stream. The processor queues the executable jobs, and applies runtime piggybacking of multiple jobs by processing workers of different types. Runtime piggybacking for a ParFOR (parallel for) ML program is optimized including configuring the runtime piggybacking based on processing worker type, degree of parallelism and minimum time thresholds.

Runtime piggybacking of concurrent jobs in task-parallel machine learning programs

A method for optimization of machine learning (ML) workloads on a graphics processor unit (GPU). The method includes identifying a computation having a generic pattern commonly observed in ML processes. An optimized fused GPU kernel is employed to exploit temporal locality for inherent data-flow dependencies in the identified computation. Hierarchical aggregation spanning a memory hierarchy of the GPU for processing for the identified computation is performed. GPU kernel launch parameters are estimated following an analytical model that maximizes thread occupancy and minimizes atomic writes to GPU global memory.

Growing community of inventors

San Jose, CA, United States of America

Berthold Reinwald