International Business Machines Corporation

Joshua M Rosenkranz

Mudhakar Srivatsa

Raghu Kiran Ganti

Ramya Raghavendra

Linsong Chu

Pranita Sharad Dewan

Seema Nagar

Mathews Thomas

Praveen Jayachandran

Bong Jun Ko

Supriyo Chakraborty

Vijay Arya

Petros Zerfos

Amitkumar Manoharrao Paradkar

Pooja Aggarwal

Chander Govindarajan

Shiqiang Wang

Utpal Mangla

Carlos Henrique Andrade Costa

Xuan-Hong Dang

Rohan R Arora

Satishkumar Sadagopan

Priyanka Prakash Naik

Kavya Govindarajan

Tuan Minh Hoang Trong

Dipanwita Guhathakurta

Christopher Streiffer

A system, computer program product, and method are provided for distributed data workflow semantics. A pipeline, such as a machine learning (ML) pipeline, is represented in a data flow graph (DFG). The represented pipeline is subject to annotations, with the annotations including pipeline nodes and object references. The pre-processed pipeline is subject to execution or processing with the annotated object references capturing object lineage. Output from the executed pipeline is constructed and a corresponding control signal is formatted to dynamically and selectively control an operatively coupled physical hardware device or software.

Annotation of a machine learning pipeline with operational semantics to support distributed lineage tracking

Aspects of the present disclosure relate to annotating or tagging customer data. In some embodiments, the annotating can include summarizing touchpoints into k-hot encoding feature vectors, mapping the feature vectors onto an embedding layer, predicting a hierarchical data sequence using the embedding layer and the feature vectors, extracting the feature vectors that are most influential in predicting the embedding layer, and outputting the touchpoints associated with the most influential feature vectors.

Annotating customer data

An embodiment includes detecting a set of anomalies recorded during a first predefined window of time in log entries for a computer environment. The embodiment also includes generating cluster data representative of a cluster of anomalies from among the set of anomalies, where the cluster is formed using a lattice clustering algorithm that spatially distinguishes the cluster of anomalies from other anomalies in the set of anomalies. The embodiment also includes composing an explanation using log templates generated from log entries associated with the cluster of anomalies.

Generating incident explanations using spatio-temporal log clustering

Data associated with performances of microservices functioning in a distributed computing environment is clustered by executing an unsupervised machine learning algorithm. A representative data is selected from a cluster, selecting performed for a plurality of the clusters. Based on time series data of the representative data associated with the plurality of the clusters, causal extraction is performed. Based on the causal extraction and the plurality of the clusters, a causal graph is constructed. The causal graph is embedded into vector space. Based on the embedded vector space, an artificial neural network model can be trained for managing the distributed computing environment.

Unsupervised learning to simplify distributed systems management

Aspects of the invention include techniques for decomposing trend and seasonality components in a forecast of parametric time series data. A non-limiting example method includes receiving time series data that includes a plurality of values taken over a first period of time. A forecast is generated using the time series data. The forecast can include one or more predicted values over a second period of time. The forecast is decomposed into N components and 2coalitions are defined for the N components. A coalition value is determined for each coalition of the 2coalitions.

Trend and seasonality decomposition

A computer implemented method performs data skipping in a hierarchically organized computing system. A group of processor units determines leaf node data sketches for data in leaf nodes in the hierarchically organized computing system. The leaf node data sketches summarize attributes of data in the leaf nodes. The group of processor units aggregates the leaf node data sketches at intermediate nodes in the hierarchically organized computing system to form aggregated data sketches at the intermediate nodes and retains data sketches received at the intermediate nodes from a group of child nodes to form retained data sketches. The retained data sketches are one of leaf node data sketches and the aggregated data sketches. The group of processor units searches the data using the retained data sketches and the data skipping within the hierarchically organized computing system in response to queries made to the intermediate nodes in the hierarchically organized computing system.

Hierarchical data skipping using data sketches

Aspects of the present disclosure relate to identifying friction points in customer data. In some embodiments, identifying friction points can include receiving a set of input sequence data and predicted class labels for the set of input sequence data; selecting input sequences, from the set of input sequence data, that have class labels matching a ground truth class label; reducing the selected sequences to anchor points; and grouping the reduced selected sequences into critical data set signatures using discriminatory subsequence mining.

Identifying friction points in customer data

An ensemble of autoencoder models can be trained using different seeds. The trained ensemble of autoencoder models can be run on new time series data to generate a prediction associated with the new time series data. The new time series data can include multiple dimensions per time step. Reconstruction errors can be determined for the prediction. Dimensions having highest reconstruction errors can be selected among the multiple dimensions based on a threshold. The prediction can be segmented based on bursts of the reconstruction errors over time, where temporal segments can be obtained. At least one common pattern including a set of dimensions among the selected dimensions across the temporal segments can be obtained to represent a failure fingerprint.

Localizing faults in multi-variate time series data

Method and apparatus for exchanging corpora via a data broker are provided. One example method generally includes receiving, at the data broker from a holder of a first corpus application, a coreset for the first corpus and transmitting the coreset to a set of data providers. The method further includes receiving, from a first data provider of the set of data providers, a value with respect to the coreset of a second corpus associated with the first data provider and transmitting, from the data broker to the holder of the first corpus, the value. The method further includes receiving, at the data broker from the holder of the first corpus, a request to receive the second corpus and receiving the second corpus from the first data provider. The method further includes validating the value of the second corpus and transmitting the second corpus to the holder of the first corpus.

Determining value of corpora for machine learning using coresets

Aspects of the invention include generating a vector representation of a root node of the error based on a hierarchical topology of a computing system; generating a respective vector representations of each subject matter expert of a plurality of subject matter experts based at least in part on the hierarchical topology; selecting a subject matter expert based at least in part on the vector representation of root cause of the error; and uploading a diagnostic software to the computing system.

Reinforcement learning approach to root cause analysis

A request may be received to join one or more attributes of at least two independent sets of data into a data structure. The one or more attributes may include a time attribute. The two independent sets of data may be included within a data store. It may be determined that there are one or more null values associated with the join to the data structure. In response to at least the determining that there are one or more null values associated with the join, one or more values may be imputed into one or more fields corresponding to the one or more null values, wherein there are no null values in the one or more fields subsequent to the imputing.

Growing community of inventors

White Plains, NY, United States of America

Joshua M Rosenkranz