International Business Machines Corporation

Globalfoundries Inc.

University of Florida Research Foundation, Incorporated

William Marsh Rice University

Peter Haas

Guy Maring Lohman

John Sismanis

Tanveer Fathima Syeda-Mahmood

John Michael Lake

Volker Gerhard Markl

Wang-Chiew Tan

Fei Xu

Rainer Gemulla

Vuk Ercegovac

Kevin Scott Beyer

Berthold Reinwald

Andrey Balmin

Walid Rjaibi

Ignacio Gabriel Terrizzano

Paul Geoffrey Brown

Ronald Mak

Vijayshankar Raman

Mir Hamid Pirahesh

Eugene Jon Shekita

Wolfgang Lehner

Paul Philip Maglio

Yinan Li

Patricia G Selinger

Ravindranath Jampani

Luis L Perez

Nimrod Megiddo

Ling Wang

Sam Sampson Lightstone

Fatma Ozcan

Yuanyuan Tian

Ariel Fuxman

Cheryl Ann Kieliszewski

Ivan Popivanov

Ashraf Aboulnaga

Jan Vondrak

John (Yannis) Sismanis

Liping Peng

Christopher M Jermaine

Mingxi Wu

Yannis Sismanis

Wenlei Xie

Mohamed Eltabakh

Calisto Paul Zuzarte

Vanja Josifovski

David Everett Simmen

Michael Jeffrey Winer

Lin Qiao

Chun Zhang

Frederick Ralph Reiss

Marcel Kutsch

Ashutosh Singh

Markos Zaharioudakis

Ihab F Ilyas

John E Lumby

Nicole C Barberis

Piyaphol Phoungphol

Subramanian Arumugam

Chistopher Matthew Jermaine

Ashraf Ismail Aboulnaga

Vijayashankar Raman

Felix Endres

Ron Mak

Utkarsh Srivastava

Ashutosh Vir Vikram Singh

Systems and associated methods for highly parallel processing of parameterized simulations are described. Embodiments permit processing of stochastic data-intensive simulations in a highly parallel fashion in order to distribute the intensive workload. Embodiments utilize methods of seeding records in a database with a source of pseudo-random numbers, such as a compressed seed for a pseudo-random number generator, such that seeded records may be processed independently in a highly parallel fashion. Thus, embodiments provide systems and associated methods facilitating quicker data-intensive simulation by enabling highly parallel asynchronous simulations.

Systems and methods for highly parallel processing of parameterized simulations

Embodiments of the present invention relate to elimination of blocks such as splits in distributed processing systems such as MapReduce systems using the Hadoop Distributed Filing System (HDFS). In one embodiment, a method of and computer program product for optimizing queries in distributed processing systems are provided. A query is received. The query includes at least one predicate. The query refers to data. The data includes a plurality of records. Each record comprises a plurality of values in a plurality of attributes. Each record is located in at least one of a plurality of blocks of a distributed file system. Each block has a unique identifier. For each block of the distributed file system, at least one value cluster is determined for an attribute of the plurality of attributes. Each value cluster has a range. The predicate of the query is compared with the at least one value cluster of each block. The query is executed against only those blocks where the predicate is met by at least one value cluster.

Split elimination in mapreduce systems

An experiment manager is discussed for the design and execution of numerical experiments in composite simulation models, such as those created using the Smarter Planet Platform for Analysis Simulation of Health (Splash). The experiment manager independently elicits experiment-related information from each contributor of a component model, and uses this information to subsequently assist the creator of a composite model in selecting experimental factors, creating experimental designs based on these factors, and executing the experiments. This functionality permits cross-disciplinary modeling, simulation, sensitivity analysis and optimization in the setting of complex systems.

System for design and execution of numerical experiments on a composite simulation model

An aspect of synchronization of time between simulation models includes receiving a specification request for generating a set of target time-series data using a set of source time-series data, obtaining specification information relating to the set of source time-series data, obtaining specification information relating to the set of target time-series data and obtaining the source time-series data; comparing the source specification information and the target specification information to determine if the set of source time-series data are time-aligned with the set of target time-series data, and converting the set of source time-series data to the set of target time-series data upon determination that time alignment is needed.

Synchronization of time between different simulation models

A computer-implemented method, according to one embodiment, includes: generating two or more sample graphs by sampling edges of a current snapshot of a dynamic graph, generating two or more partial results by executing an algorithm on the two or more sample graphs, combining the partial results into a final result, and incrementally maintaining the sample graphs. Edges included in the current snapshot of a dynamic graph and which were added to the dynamic graph in a most recent update thereto are included in each of the generated two or more sample graphs. Moreover, incrementally maintaining the sample graphs includes: subsampling each of the edges of each of the sample graphs at a given time by applying a Bernoulli trial, and combining a result of the subsampling with new edges received in a batch corresponding to the given time to form new sample graphs.

Dynamic interaction graphs with probabilistic edge decay

In one general embodiment, a computer-implemented method is provided for analyzing a dynamic graph. The computer-implemented method includes generating two or more sample graphs by sampling edges of a current snapshot of a dynamic graph. Additionally, the computer-implemented method includes generating two or more partial results by executing an algorithm on the sample graphs. Still yet, the computer-implemented method includes combining the partial results, from executing the algorithm on the sample graphs, into a final result.

Split elimination in MapReduce systems

Embodiments of the present invention provide a method for result caching to improve the statistical efficiency of a composite model comprising a first component model and at least one other component model. The method comprises developing at least one metamodel for at least one component model of the composite model, computing a re-use factor that maximizes an asymptotic efficiency measure based on the metamodel, determining a number of executions to run for the composite model, and determining a number of executions to run for the first component model based on the re-use factor and the number of executions to run for the composite model. The number of executions to run for the first component model is fewer than the number of executions to run for the composite model. Each output generated from each execution of the first component model is cached and provided as input to another component model.

Result caching for improving statistical efficiency of composite simulation models

An aspect of handling a plurality of simulated models includes receiving from a first simulation model a stream of source data and determining data processing compatibility between the simulation model and a second-simulation model by i) obtaining metadata relating to the first-simulation models from a memory and ii) obtaining metadata relating to the second simulation model from the memory. An aspect also includes generating a transformation component using a schema mapping tool for providing compatibility between the first and second simulation models upon a determination that the compatibility does not exist between the two models. The&#x2014;transformation component is generated&#x2014;using the metadata relating to the first and second simulation models. The method also includes generating a&#x2014;stream of target data by the second simulation model via the transformation component. The second simulation model uses the stream of source data as input.

Composite simulation modeling and analysis

An aspect of combining simulation models includes a processor configured to process information relating to a plurality of simulated models. The processor collects the information to be processed related to the simulated models in at least one memory having metadata relating to the simulated models. The processor has a transformation component generated using a schema mapping tool. The transformation component detects and corrects any incompatibility between a first and a second simulation model by obtaining metadata relating to the first simulation model and the second simulation model from the memory.

Stratified sampling of a plurality of records is performed. A plurality of records are partitioned into a plurality of splits, wherein each split includes at least a portion of the plurality of records. The split of the plurality of splits is provided to a mapper. The mapper assigns at least a portion the records of the at least one split to a group based on a strata of the assigned records, and filters the records of the group based on a comparison of the weights of the records to a local threshold of the mapper. The mapper updates the local threshold of the mapper by communicating with a coordinator. The mapper shuffles the group to a reducer, where the reducer filters the records of the group based on the weights of the records. The reducer provides a stratified sampling of the plurality of records based on the group.

Stratified sampling using adaptive parallel data processing

A computer-implemented method includes partitioning a plurality of records into a plurality of splits. Each split includes at least a portion of the plurality of records. The method further includes providing at least one split of the plurality of splits to a mapper. The mapper scans the input data set, transforms each input record using a map function, and extracts a grouping key in parallel. The method further includes assigning at least a portion the records of the at least one split to a group. Each assignment to the group is based on a strata of the assigned record, and filtering the records of the group. Each filtering is based on a comparison of a weight of a record to a local threshold of the mapper. The method further includes shuffling the group to a reducer and providing a stratified sampling of the plurality of records based on the group.

Embodiments relate to a method and computer program product for generating a composite simulated model. A method includes receiving a specification request for generating a set of target time-series data from a set of source time-series data and obtaining specification information relating to the set of source time-series data, obtaining specification information relating to the set of target time-series data, and obtaining the source time-series data. The method also includes comparing the source and target specification information to determine if the set of source time-series data are time-aligned with the set of target time-series data and converting the set of source time-series data to the set of target time-series data upon determination that time alignment is needed.

Growing community of inventors

San Jose, CA, United States of America

Peter Haas