International Business Machines Corporation

North Carolina State University

Daniel Dean

Anca Sailer

Hui Lei

Yu Gu

Jing Min Xu

Yaoping Ruan

Xiao Xi Liu

Alexei A Karve

Pei Ni Liu

Andrzej P Kochut

Charles Otto Schulz

Dingcheng Li

Xiaohui Gu

Checed A Rodgers

Dingcheng Lil

Performing an operation comprising transforming an input dataset to a predefined format, extracting, from the transformed dataset, a plurality of features describing the transformed dataset, and generating, by a machine learning (ML) algorithm executing on a processor and based on an ML model, a plurality of rules for modifying the transformed dataset to conform with a first data model.

Cognitive data discovery and mapping for data onboarding

Embodiments generally relate transforming data for a target schema. In some embodiments, a method includes receiving input data, where the input data includes a plurality of segments, and where the segments include a plurality of source fields containing target data. The method further includes characterizing the input data based at least in part on a plurality of predetermined metrics, where the predetermined metrics determine a structure of the input data. The method further includes mapping the target data in the source fields of the segments to a plurality of target fields of a target schema based at least in part on the characterizing. The method further includes populating the target fields of the target schema with the target data from the source fields based at least in part on the mapping.

Transforming data for a target schema

A mechanism is provided for implementing operation optimization. Responsive to a request to load data via a input/output (I/O) load operation, prior to loading the data, transformed data in a local memory is searched for a match to the requested data. Responsive to identifying transformed that matches the requested data, the transformed data in the local memory is used to fulfill the request. Responsive to failing to identify transformed data in the local memory that matches the requested data, untransformed data in the local memory is searched for a match to the requested data. Responsive to identifying untransformed data that matches the requested data, the untransformed data in the local memory is used to fulfill the request. Responsive to failing to identify untransformed data in the local memory that matches the requested data, the requested data may be loaded from the storage system via the I/O load operation.

Optimizing data retrieval operation in big-data processing systems

Efficient data retrieval in big-data processing systems

An unsupervised behavior learning system and method for predicting anomalies in a distributed computing infrastructure. The distributed computing infrastructure includes a plurality of computer machines. The system includes a first computer machine and a second computer machine. The second computer machine is configured to generate a model of normal and anomalous behavior of the first computer machine, where the model is based on unlabeled training data. The second computer machine is also configured to acquire real-time data of system level metrics of the first machine; determine whether the real-time data is normal or anomalous based on a comparison of the real-time data to the model; and predict a future failure of the first computer machine based on multiple consecutive comparisons of the real-time data to the model. Upon predicting a future failure of the first computer machine, generate a ranked set of system-level metrics which are contributors to the predicted failure of the first computer machine, and generate an alarm that includes the ranked set of system-level metrics. The model of normal and anomalous behavior may include a self-organizing map.

Unsupervised behavior learning system and method for predicting performance anomalies in distributed computing infrastructures

A computer executes a first version of a code module in a first test environment, collects a first set of execution measurements, and creates a first profile based on the first set of execution measurements. The computer executes the first version of the code module in a second test environment, collects a second set of execution measurements, and creates a second profile based on the second set of execution measurements. In response to a difference between the first profile and the second profile exceeding a threshold, the computer provides diagnostic data characterizing the difference, analyzes an impact of the difference on the execution of the code module, calculates a risk of code module failure value based on analyzing the impact of the difference, and provides a recommended action based on analyzing the impact of the difference and the calculated risk of code module failure value.

Early risk identification in DevOps environments

Growing community of inventors

Yorktown Heights, NY, United States of America

Daniel Dean