Big data distributed processing and secure data transferring with resource allocation and rebate

Abstract

Aspects of the disclosure relate to resource allocation and rebating during in-flight data masking and on-demand encryption of big data on a network. Computer machine(s), cluster managers, nodes, and/or multilevel platforms can request, receive, and/or authenticate requests for a big data dataset, containing sensitive and non-sensitive data. Profiles can be auto provisioned, and access rights can be assigned. Server configuration and data connection properties can be defined. Secure connection(s) to the data store can be established. Sensitive information can be redacted into a sanitized dataset based on one or more data obfuscation types. RAM requirements and current RAM allocation can be diagnosed. Portion(s) of the current RAM allocation exceeding the RAM requirements can be rebated. The encrypted data can be transmitted, in response to the request, to a source, a target, and/or another computer machine and can be decrypted back into the sanitized dataset.