Hewlett Packard Enterprise Development LP

Xiongbing Ou

Swami Viswanathan

Kartik Mathur

Vadim Finkelstein

Thomas A Phelan

Joel Baxter

David E Lee

David B Lee

Ramaswami Kishore

Lakshminarayanan Gunaseelan

Lakshm Inarayanan Gunaseelan

Example implementations relate to a role-based autoscaling approach for scaling of nodes of a stateful application in a large scale virtual data processing (LSVDP) environment. Information is received regarding a role performed by the nodes of a virtual cluster of an LSVDP environment on which a stateful application is or will be deployed. Role-based autoscaling policies are maintained defining conditions under which the roles are to be scaled. A policy for a first role upon which a second role is dependent specifies a condition for scaling out the first role by a first step and a second step by which the second role is to be scaled out in tandem. When load information for the first role meets the condition, nodes in the virtual cluster that perform the first role are increased by the first step and nodes that perform the second role are increased by the second step.

Autoscaling nodes of a stateful application based on role-based autoscaling policies

Secured data access in virtual data processing is described. An example includes instructions to receive a request from an application in a compute node of a compute cluster in a virtual data processing environment to access a secured data source for a user, the virtual data processing environment including a multiple secured data sources that are accessible by compute nodes of the virtual compute cluster; fetch a credential in a current application context and forward the credential for validation; validate the credential with a credential authority; and, upon successfully validating the credential, authenticate the user at the secured data source and establish a connection with the secured data source.

Secured data access in virtual data processing

Embodiments described herein are generally directed to caching and data access improvements in a large scale data processing environment. According to an example, an agent running on a first worker node of a cluster receives a read request from a task. The worker node of the cluster to which the data at issue is mapped is identified. When the first worker node is the identified worker node, it is determined whether its cache contains the data; if so, the data is fetched from a remote data lake and the agent locally caches the data; otherwise, when the identified worker node is another worker node of the compute cluster, the data is fetched from a remote agent of that worker node. The agent responds to the read request with cached data, data returned by the remote data lake, or data returned by the remote data agent as the case may be.

Efficient caching and data access to a remote data lake in a large scale data processing environment

Described herein are systems, methods, and software to migrate virtual nodes of a data processing cluster. In one implementation, a management system monitors an executing data processing cluster on one or more first hosts to determine when the data processing cluster satisfies migration criteria. Once satisfied, the management system selects one or more second hosts to support the data processing cluster based on accommodation data associated with the hosts. After selection, the management system may initiate operations to migrate the data processing cluster from the one or more first hosts to the one or more second hosts.

Enhanced migration of clusters based on data accessibility

Described herein are systems, methods, and software to enhance the management of virtual nodes in a computing environment. In one example, a method includes identifying a transition event for a first version of a virtual node executing on a first host, identifying a second host to support a second version of the virtual node, and initiating execution of the second version of the virtual node on the second host. In some implementations, the virtual node comprises a non-persistent portion on the executing host and a persistent portion on a second storage element.

Growing community of inventors

San Jose, CA, United States of America

Xiongbing Ou