International Business Machines Corporation

Vasileios Vasileiadis

Michail Vlachos

Charalampos Pozidis

Thomas P Parnell

Michael Johnston

Reinhard Wolfram Heckel

Nikolas Ioannou

Kubilay Atasu

Srikumar Venugopal

Christian Pinto

Ioannis Gkoufas

Celestine Duenner

Cesar Berrospi Ramis

A computer implemented method for optimizing performance of workflow includes associating each of a plurality of workflow nodes in a workflow with a data cache and managing the data cache on a local storage device on one of one or more compute nodes. A scheduler can request execution of the tasks of a given one of the plurality of workflow nodes on one of the one of more compute nodes that hosts the data cache associated with the given one of the plurality of workflow nodes. Each of the plurality of workflow nodes is permitted to access a distributed filesystem that is visible to each of the plurality of compute nodes. The data cache stores data produced by the tasks of the given one of the plurality of workflow nodes.

Optimization of workflows with dynamic file caching

Embodiments are provided for providing workflow pipeline optimization in a computing environment. Execution of a workflow containing dependencies between one or more subject nodes and one or more observer nodes may be dynamically optimized by determining a wait time between successive executions of the workflow for the one or more observer nodes.

Workflow pipeline optimization based on machine learning operation for determining wait time between successive executions of the workflow

Two sets Xand Xof histograms of words, and a vocabulary V are accessed. Each of the two sets is representable as a sparse matrix, each row of which corresponds to a histogram. Each histogram is representable as a sparse vector, whose dimension is determined by a dimension of the vocabulary. Two phases compute distances between pairs of histograms. The first phase includes computations performed for each histogram and for each word in the vocabulary to obtain a dense, floating-point vector y. The second phase includes computing, for each histogram, a sparse-matrix, dense-vector multiplication between a matrix-representation of the set Xof histograms and the vector y. The multiplication is performed to obtain distances between all histograms of the set Xand each histogram X[j]. Distances between all pairs of histograms are obtained, based on which distances between documents can subsequently be assessed.

Low-complexity methods for assessing distances between pairs of documents

Methods and apparatus are provided to determine entities and attributes dependencies for creating recommendations of items or entities using a highly scalable architecture. For example, a user may be recommended an item if a probability model of the method determines that the user relates to the item although the user has no contact to the item before the method is performed. The methods and apparatus provide a data structure representing a matrix having rows representing entities and columns representing attributes of the entities. Each entity of the entities of the data structure may include a user and each attribute of the attributes of the data structure may include an item. A cell of the matrix may be formed by a component pair including an entity and an attribute. In this manner, the methods and apparatus provide an efficient way for processing the probability model.

System and method for a scalable recommender system using massively parallel processors

Embodiments include processing a data structure representing a dependency matrix having columns representing respective first components and rows representing respective second components. Aspects include assigning each cell of the matrix a value indicative of the level of dependency or indicative of an unknown dependency of a pair of first and second components forming the cell and assigning each component of the first and second components an affiliation vector indicative of the strength of affiliation of the component to N predefined initial clusters of cells of the matrix. Aspects also include determining a probability model using the affiliations vectors parameters and estimating the parameters of the probability model for a plurality of different numbers of clusters starting from the initial number N of clusters. Aspects further include computing a score for the parameters of the probability model estimated and selecting the parameters of the probability model with the highest computed score.

Growing community of inventors

Waedenswil, Switzerland

Vasileios Vasileiadis