International Business Machines Corporation

Xiang Ni

Kun-Lung Wu

Scott A Schneider

Lingfei Wu

Michael John Witbrock

Shen Li

Xavier Rene Guérin

Embodiments for automating multidimensional elasticity for streaming applications in a computing environment. Each operator in a streaming application may be identified and assigned into one of a variety of groups according to similar performance metrics. One or more threading models may be adjusted for one or more of the groups to one or more different regions of the streaming application.

Automated multidimensional elasticity for streaming application runtimes

In an approach to improve online path profiling by identifying heavy trafficked paths in streaming applications embodiments track data tuples through a streaming application. Embodiments consist of periodically executing a tuple submission application programming interface (API) and a tuple processing API. Additionally, embodiments record active paths using a profiler thread by taking snapshots of a current path index across threads. Furthermore, embodiments generate path usage information based on the recorded active path, generate a path usage graph based on the path usage information; and optimize identified trafficked paths based on the path usage data and path usage map.

Path profiling for streaming applications

A method and system of creating a model for large scale data analytics is provided. Training data is received in a form of a data matrix X and partitioned into a plurality of partitions. A random matrix T is generated. A feature matrix is determined based on multiplying the partitioned training data by the random matrix T. A predicted data {tilde over (y)} is determined for each partition via a stochastic average gradient (SAG) of each partition. A number of SAG values is reduced based on a number of rows n in the data matrix X. For each iteration, a sum of the reduced SAG values is determined, as well as a full gradient based on the sum of the reduced SAG values from all rows n, by distributed parallel processing. The model parameters w are updated based on the full gradient for each partition.

Efficient large-scale kernel learning using a distributed processing architecture

A multithreading system that performs elastic threading and dynamic patching is provided. The system receives a compiled object of a computing process, the compiled object comprising a set of locking instructions for ensuring exclusive access of a resource by the computing process. The system determines a thread count for the computing process. When the thread count indicates that a single thread is allocated to execute the computing process, the system patches the compiled object with a set of no-operation (NOP) instructions in place of the set of locking instructions. When the thread count indicates that two or more threads are allocated to execute the computing process, the system patches the compiled object with the set of locking instructions in place of the set of NOP instructions. The system executes the computing process according to the patched compiled object.

Growing community of inventors

Yorktown Heights, NY, United States of America

Xiang Ni