Big data distributed processing and secure data transferring with obfuscation

Abstract

Aspects of the disclosure relate to in-flight data masking and on-demand encryption of big data on a network. Computer machine(s), cluster managers, nodes, and/or multilevel platforms can request, receive, and/or authenticate requests for a big data dataset, containing sensitive and non-sensitive data, in a data store based on credentials received from a source. Profiles can be auto provisioned, and access rights can be assigned. Server configuration and data connection properties can be defined. A secure connection to the data store can be established. The sensitive information in the big data dataset can be redacted into a sanitized dataset based on one or more data obfuscation types. The encrypted data can be transmitted, in response to the request, to a source, a target, and/or another computer machine and can be decrypted back into the sanitized dataset.