American Express Travel Related Services Company, Inc.

Man Chon U

Andras L Ferenczi

Rares Ioan Almasan

Swatee Singh

Mohammad N Nauman

Yan Yang

Beau Britain

Linou Zhu

Thai Chiun Huang

Ashok K Nair

Phanikalyan Cherukuri

Aman Madan

Pulkit Aggarwal

Ritesh Kumar Bansal

Deeksha Sikarwar

Annie Arora

Drishti Ohri

Disclosed herein are system, method, and computer program product embodiments for converting queries and scripts used in big data systems. A script conversion system converts a script and its queries from a first big data format to a second big data format. The script conversion system uses a configuration mapping that includes direct and/or indirect function mappings. For the direct mapping, the script conversion system replaces the function with a new function in the second big data format. For the indirect mapping, the script conversion system constructs an equivalent function in the second big data format using information extracted from parsing queries in the first big data format. The script conversion system also formats a converted query using a project mapping and/or generates a conversion report indicating successful or unsuccessful conversions. Scripts may be converted from an on-premises data warehouse system to a cloud-based data warehouse system.

Automated conversion of big data queries and scripts

A computer-based system may engineer features based on semantic types. The computer-based system may implement deep learning algorithms and derive a domain-specific feature engineering strategy from semantic type predictions and data profiling. The computer-based system may utilize embedded domain (e.g., financial industry, etc.) knowledge to generate curated features from raw data (e.g., transactional datasets, relational datasets, etc.).

Featuring engineering based on semantic types

Disclosed are various embodiments for using a distributed ledger to track the evolution or deployment of feature sets for machine-learning. An approval request to approve creation of a feature set record in a distributed ledger can be received. The approval request can include a feature set hash representing a feature set, a code hash representing a feature set generator, and a data hash representing a data set. An approval decision can be generated for the feature set record based at least in part on the approval request and at least one approval rule stored in an approval data store. The approval decision can be transmitted to a distributed agent. The approval decision can approve or reject creation of the feature set record in the distributed ledger.

Distributed ledger based feature set tracking

Disclosed are various approaches for managing the status of machine-learning models using distributed ledgers. A registration request for a machine-learning model can be received. The registration request can include a model name for the machine-learning model, a version identifier for the machine-learning model, a network address from which the machine-learning model can be retrieved, a source code hash for a source code version of the machine learning model, and a runtime hash for a binary executable version of the machine-learning model. A registration identifier can then be created based at least in part on the source code hash and the runtime hash. Subsequently, an entry in the distributed ledger can be created for the machine-learning model. The entry can include the registration identifier, the model name, the model version, the network address, the source code hash, and the runtime hash.

Distributed ledger based machine-learning model management

Feature engineering based on semantic types

Disclosed are various embodiments for using a distributed ledger to track the evolution or deployment of feature sets for machine-learning. A registration request is received from a first node of a distributed ledger, the registration request comprising a code hash representing a feature set generator and a data hash representing a data set. The registration request is then relayed to a second node of the distributed ledger for approval by the second node. Next, an approval for the registration request is received from the second node. Subsequently, an entry in the distributed ledger comprising the code hash and the data hash is created.

Growing community of inventors

Sunrise, FL, United States of America

Man Chon U