International Business Machines Corporation

Yuan-Chi Chang

Chung-Sheng Li

John Richard Smith

Lawrence David Bergman

Matthew Leon Hill

Mustafa Canim

Vittorio R Castelli

Long H Vu

Venkata Nagaraju Pavuluri

Wei Tan

Liana Liyow Fong

Deepak Srinivas Turaga

Mohammad Sadoghi Hamedani

Min Wang

Christian Alexander Lang

Timothy Rea Dinger

Philip Shilung Yu

Sriram Kolinjivadi Padmanabhan

Lipyeow Lim

Ioana Roxana Stanoi

Lingtao Cao

Zhen Zhang

Charu Chandra Aggarwal

Anton Viktorovich Riabov

Bishwaranjan Bhattacharjee

Nagui Halim

Timothy Ray Malkemus

George Andrei Mihaila

Octavian Udrea

Kenneth Andrew Ross

Shirin Sohrabi Araghi

Mark David Feblowitz

Jimeng Sun

Jason L Crawford

Feifei Li

Edward John Pring

Saket Sathe

Spyridon Papadimitriou

Bassem Makni

Howard Chun Chin

Carsten E Mytroen

Stuart S Horn

Edward W Shay

Fang Yuan

Peter Zimmer

Manoj N Kumar

Kun-Lung Wu

Robert G Farrell

Akiko Suzuki

Biplav Srivastava

Vibha Singhal Sinha

Zon-Yin Shae

Lawrence H Koved

Milind R Naphade

Hui-I Hsiao

Sharon Mary Trewin

Oktie Hassanzadeh

Dharmashankar Subramanian

David Mun-Hien Choy

Debdoot Mukherjee

Liangliang Cao

Jy-Jine James Lin

Marc Hubert Willebeek-LeMair

Anant Deep Jhingran

Murray Scott Campbell

Sonia Mariette Embid Droz

Gene Y C Fuh

Debarun Bhattacharjya

Ruben Rodriguez Torrado

Sharon C Adler

Quoc-Bao Nguyen

Robin Williams

Giorgio De Paola

Cristina Ibanez-Llano

Rodrigue Ngueyep Tzoumpe

Hidayet Aksu

Igor Sominsky

Richard Han

Mirella M Moro

Richard Yeh-whei Han

Raju Pavuluri

Embodiments for implementing enhanced ensemble model diversity and learning by a processor. One or more data sets may be created by combining one or more clusters of data points of a minority class with selected data points of a majority class. One or more ensemble models may be created from the one or more data sets using a supervised machine learning operation. An occurrence of an event may be predicted using the one or more ensemble models.

Enhanced ensemble model diversity and learning

A mechanism is provided for implementing a model update mechanism to update new models in real time while avoiding data loss and system downtime. Responsive to receiving a request to update a scorer model currently being executed by an existing worker thread in the data processing system, the model update mechanism initializing a new worker thread. The model update mechanism loads an updated scorer model into the new worker thread and initializes a state transfer from the existing worker thread to the new worker thread. The model update mechanism executes the updated scorer model such that the updated scorer model scores the input data. The model update mechanism then outputs a prediction based on the updated scorer model processing of the input data.

Data loss machine learning model update

Mechanism are provided to select a machine learning model from an analytics model library based on ingested data. One or more pieces of clarified data are fused to provide time-correlated data tuples of data streams. One or more features are extracted from the time-correlated data tuples and scored based on a set of predetermined rules thereby generating discriminative scoring of trigger data. Utilizing the discriminative scoring of the trigger data, trigger data of a current analytics model being utilized by the data processing and one or more new analytics models from the analytics model library are scored. Responsive to the scoring of the trigger data indicating a selection of a different analytics model from the analytics model library, the current analytics model is replaced with a selected analytics model from the analytics model library such that the data processing system executes the selected analytics model.

Context driven model selection

Split an input dataset into training and test datasets; the former includes a plurality of data examples, each represented as a feature vector, and having an associated true label. Split the training dataset into a plurality of training data subsets; for each, train a corresponding machine learning model to obtain a plurality of such models, and apply same to the test dataset to obtain a plurality of predicted labels and prediction scores. For each of the plurality of examples, compute an agreement metric based on a corresponding one of the associated true labels; corresponding ones of the predicted labels; and corresponding ones of the prediction scores. Based on the computed metric, select, for at least some of the true label values, appropriate ones of the data examples to be added to a regression set. Add the appropriate ones of the data examples from the test dataset to the regression set.

Automated data and label creation for supervised machine learning regression testing

A method, computer program product, and/or computer system improves a future efficiency of a specific system. One or more processors receive multiple historical data snapshots that describe past operational states of a specific system. The processor(s) identify a time series pattern for the time series of data in the multiple historical snapshots and calculate their variability. The processor(s) then determine that the variability in a first sub-set of the time series pattern is larger than a predefined value, and determine that future values of the first set of the time series pattern are a set of non-forecastable future values. The processor(s) also determine that the variability in a second sub-set of the time series pattern for the data is smaller than the predefined value, and utilizes this second sub-set to modify the specific system at a current time.

Modifying a particular physical system according to future operational states

A method, system and computer program product for index maintenance in a computer system comprising a plurality of nodes, a database, and an index to the database. In one embodiment, the method comprises, for a defined period of time, building a snapshot of selected change requests received by the nodes to change the database. After this defined period of time, a selection is made, based on specified criteria, whether to rebuild a new index to the database, or to add entries to a current index. When the selection is to rebuild a new index, the new index is rebuilt based on data in the database and in the change requests in the snapshot. When the selection is to add entries to a current index, entries are added to the current index based on data in the database and in the change requests in the snapshot.

Index maintenance based on a comparison of rebuild vs. update

A resource exploitation management system, method and a computer program product therefor. A description of new geological evidence for a geological resource is received, e.g., as one or more triples describing the evidence. Keywords in the description are matched against keywords in representations in a geological resource database. Geological relations are inferred from the descriptions and matched against predefined geological relations from the geological resource database. Consistent triple matches are merged with the geological resource database. The confidence level for merged matches is updated in the geological resource database.

Resource exploitation management system, method and program product

A mechanism is provided to identify wall-clock time reference dependency in one or more software components of a data analytics solution. The data analytics solution is decomposed into a set of software components. A first software component of the set of software components is deployed to a first computer server and the remaining software components are deployed to a second computer server. A system clock time on the first computer server is changed to differ from the system clock of the second computer server. Based on executing a test on the data analytics solution, a determination is made of whether the first software component, is wall-clock time independent. Responsive to the test of the of the software component failing indicating that the wall-clock time of the software component is dependent of the system clock time difference, the software component is recorded as wall-clock time dependent and an administrator is notified.

Testing and modifying calendar and event sensitive timer series data analytics

Techniques for scenario planning are provided. In one example, a computer-implemented method can comprise analyzing, by a device operatively coupled to a processor, content using a topic model. The content can be associated with a defined source and is related to one or more current events. The computer-implemented method can also comprise determining, by the device, one or more portions of the analyzed content that are relevant to one or more key risk drivers using a risk driver model. The computer-implemented method can also comprise aggregating, by the device, the determined one or more portions into one or more emerging storylines based on values of one or more attributes of the topic model.

Scenario planning and management

Scenario planning and risk management

Conventionally, in addition to indexing, a synopsis of a base table of a database is used to skip and compress data. However, scanning of the entire synopsis for all queries is required, which takes a long time when the synopsis gets significantly big in a large data warehouse. A method for efficient data skipping and compression through vertical partitioning of data is provided to eliminate the cost of synopsis storage overhead while enabling the synopsis search functionality.

Data skipping and compression through partitioning of data

Systems, methods and articles of manufacture are disclosed for building and executing analytics solutions. Such a solution may provide a comprehensive analytics solution (e.g., a risk assessment, fraud detection solution, dynamic operational risk evaluations, regulatory compliance assessments, etc.). The analytics solution may perform an analytics task using operational data distributed across a variety of independently created and governed data repositories in different departments of an organization. A framework is disclosed which allows a user (e.g., a risk analyst) to compose analytical tools that can access data from a variety of sources (both internal and external to an enterprise) and perform a variety of analytic functions.

Analytics integration workbench within a comprehensive framework for composing and executing analytics applications in business level languages

A method for searching the top-K simple shortest paths between a specified source node and a specified target node in a graph, with graph data partitioned and distributed across a plurality of computing servers, the method including a parallel path search initialized from either one or both of the source and target nodes and traversing the graph by building likely path sequences for a match. Each computing server determines and forwards a path sequence as discovery progresses until the top-K paths are discovered.

Parallel top-K simple shortest paths discovery

A method for processing a dataset in a partitioned distributed storage system having data stored in a base table and an index stored in an index table, may include receiving base and index table metadata from the partitioned distributed storage system, where the base and index table metadata includes respective table partition information. The method may further include partitioning the dataset into a set of base-delta files according to the base table metadata, and generating a set of index-delta files corresponding with the base-delta files according to the index table metadata. The method may additionally include updating the partitioned distributed storage system with the set of base-delta and the set of index-delta files, where a first update of the base table is synchronous with a second update of the index table.

Partition-based index management in hadoop-like data stores

Growing community of inventors

New York, NY, United States of America

Yuan-Chi Chang