Big data distributed processing and secure data transferring with hyper fencing

Abstract

Aspects of the disclosure relate to resource allocation and rebating during in-flight data masking and on-demand encryption of big data on a network. Computer machine(s), cluster managers, nodes, and/or multilevel platforms can request, receive, and/or authenticate requests for a big data dataset, containing sensitive and non-sensitive data. Profiles can be auto provisioned, and access rights can be assigned. Server configuration and data connection properties can be defined. Secure connection(s) to the data store can be established. The big data dataset can be uncompressed based on a codec and uncompressed data blocks can be distributed for processing. Sensitive information can be redacted into a sanitized dataset based on one or more data obfuscation types. The encrypted data can be transmitted, in response to the request, to a source, a target, and/or another computer machine and can be decrypted back into the sanitized dataset.