International Business Machines Corporation

Wei Lei Wu

Deyu Wang

Ming Liang Guo

Yue Liu

Yan Zhao Liu

Fei Qi

Mechanisms are provided for managing caching of data in a multi-tenant cache storage device utilized by a distributed dataset based application. The mechanisms retrieve Resilient Distributed Dataset (RDD) block size information for an RDD associated with an application. A probability distribution of RDD block sizes is generated based on the RDD block size information and a maximum size for a RDD block of data is estimated based on the probability distribution. An amount of free space in a portion of the multi-tenant cache storage device allocated to the application is estimated based on the estimated maximum size for the RDD block of data. Cache operations for caching data associated with the application to the multi-tenant cache storage device are managed based on the estimated amount of free space in the portion of the multi-tenant cache storage device allocated to the application.

Caching a block of data in a multi-tenant cache storage device based on space usage boundary estimates

A computer-implemented method, system and computer program product for protecting data being transferred during a session with an application are presented. In response to receiving a session request that includes an application identifier for an application, a data processing node establishes a session with the application and assigns a session identifier for that session. A session response, which includes the session identifier, to the session request is sent. In response to receiving a first request that includes the session identifier, a first pipe in the session is established. This first pipe enables data related to the first request to be isolated according to an assigned first pipe identifier for the first pipe. A first response to the first request is sent, wherein the first response comprises the session identifier and the first pipe identifier.

Method and system for data processing

A computer-implemented method for protecting data being transferred during a session with an application is presented. In response to receiving a session request that includes an application identifier for an application, a data processing node establishes a session with the application and assigns a session identifier for that session. A session response, which includes the session identifier, to the session request is sent. In response to receiving a first request that includes the session identifier, a first pipe in the session is established. This first pipe enables data related to the first request to be isolated according to an assigned first pipe identifier for the first pipe. A first response to the first request is sent, wherein the first response comprises the session identifier and the first pipe identifier.

Growing community of inventors

Beijing, China

Wei Lei Wu