Salesforce Com, Inc.

Tappingstone Inc.

Kit Pang Szeto

Simon Chan

Ka Hou Chan

Karl Ryszard Skucha

Jean-Marc Soumet

Matvey Tovbin

Yue Kwen Justin Yip

Emmanual Felipe Oliveira

Yan Yang

Eli Levine

Vitaly Gordon

Gregory H Rice

Ming-Yang Chen

Leah McGuire

Ka Chun Au

Johnson Liu

Vlad Patryshev

Christopher James Wu

Shaun Senecal

Qiong Liu

Thomas Gerber

Ji Oh Yoo

Bilong Chen

Marco Vivero

Joshua Sauter

Techniques and architectures for data modeling and management. Data modeling services are provided to agents within multiple different operating environments of a computing environment having at least one database stored on one or more physical memory devices communicatively coupled with one or more hardware processors the one or physical memory devices. Building and versioning of data modeling projects is coordinated and data utilized for the data modeling projects with the one or more hardware processors.

Techniques and architectures for managing disparate heterogeneous cloud-based resources

Methods, systems, and devices for data access and processing are described. To set up secure environments for data processing (e.g., including machine learning), an access control system may first receive approval from an authorized user (e.g., an approver) granting access to data objects in a multi-tenant data store. The system may determine tenant-specific paths for retrieving the data objects from the data store, and may initialize a number of virtual computing engines for accessing the data. Each computing engine may be tenant-specific based on the path(s) used by that computing engine, and each may include an access role defining the data objects or data object types accessible by that computing engine. By accessing the requested data objects according to the tenant-specific path prefixes and access roles, the virtual computing engines may securely maintain separate environments for different tenants and may only allow user access to approved tenant data.

Securely accessing and processing data in a multi-tenant data store

A system may generate a score for a predictive model based on receiving a streaming data flow of events associated with a predictive model for a tenant. The system may receive the streaming data flow and calculate one or more feature values in real time based on the reception. The system may store each of the calculated features to a multi-tenant database server. The system may calculate a score for the predictive model based on the storage and may transmit an indication of the score (e.g., a prediction) based on the calculation. The system may transmit the score to, for example, a computing device.

Feature generation and storage in a multi-tenant environment

In accordance with disclosed embodiments, there are provided systems, methods, and apparatuses for implementing machine learning model training and deployment with a rollback mechanism within a computing environment. For example, an exemplary machine learning platform includes means for receiving training data as input at the machine learning platform, in which the training data includes a multiple transactions, each of the transactions specifying a plurality of features upon which to make a prediction and a label representing a correct answer for the plurality of features according to each respective transaction; specifying a model to be trained by the machine learning platform using the training data, in which the model includes a plurality of algorithms and source code; generating a new predictive engine variant by training the model to algorithmically arrive upon the label representing the correct answer as provided with the training data based on the plurality of features for each of the multiple transactions; versioning the new predictive engine variant based at least on the time the new predictive engine variant was generated a version of the source code utilized within the model and the training data received as input; deploying the new predictive engine variant into a production environment to replace a prior version of the predictive engine variant; and rolling back the new predictive engine variant from the production environment to a specified version which is less than a version of the new predictive engine variant. Other related embodiments are disclosed.

Systems, methods, and apparatuses for implementing machine learning model training and deployment with a rollback mechanism

Methods, systems, and devices for multi-tenant workflow processing are described. In some cases, a cloud platform may utilize a set of pre-defined batch processes (e.g., workflow templates) and tenant-specific configurations for instantiating and executing tenant-specific batch processes for each tenant of a user. As such, the cloud platform may utilize common data process workflows for each tenant, where a configuration specifies tenant-specific information for the common data process workflows. The workflow templates may include a set of job definitions (e.g., actions for a server to execute) and a schedule defining the frequency for running the templates for a specific project. The configurations may indicate a tenant to execute the workflow templates for, and may include tenant-specific information to override default template information. The cloud platform or a designated server or server cluster may instantiate and execute workflows based on one or more combinations of configurations and indicated workflow templates.

Template-driven multi-tenant workflow processing

In accordance with disclosed embodiments, there are provided systems, methods, and apparatuses for implementing predictive engine evaluation and replay of engine performance. An exemplary system may include, for example: means selecting a first set of one or more algorithms for a machine learning model; tuning a first group of predictive engine parameters for the machine learning model; training the machine learning model with one or more sources of data using the selected first set of one or more algorithms and the first group of tuned predictive engine parameters to generate a first predictive engine variant from the trained machine learning model; selecting a second set of one or more algorithms for a machine learning model which are different than the first set; tuning a second group of predictive engine parameters for the machine learning model which are different than the first group; training the machine learning model with the one or more sources of data using the selected second set of one or more algorithms and the second group of tuned predictive engine parameters to generate a second predictive engine variant from the trained machine learning model; performing multiple experiments using the first and second predictive engine variants; comparing results from the multiple experiments; and deploying either the first predictive engine variant or the second predictive engine variant based on the comparison of the results of the multiple experiments. Other related embodiments are disclosed.

Methods and systems for predictive engine evaluation and replay of engine performance

Disclosed are methods and systems of creating, evaluating, and tuning a predictive engine for machine learning, including steps to deploy the predictive engine with an initial parameter set; receive queries to the deployed engine variant and in response, generate predicted results; receive corresponding actual results; associate the queries, the predicted results, and the actual results with a replay tag; evaluate the performance of the deployed engine variant; generate a new engine parameter set based on tuning of one or more parameters of the initial engine parameter set, according to the evaluation results; deploy the new engine variant to replace the initial engine variant; receive a replay request from an operator specifying the currently or a previously deployed engine variant; and in response to the replay request, replay at least one of the queries, the corresponding predicted results, the actual results, and the evaluation results.

Methods and systems for predictive engine evaluation, tuning, and replay of engine performance

Growing community of inventors

Sunnyvale, CA, United States of America

Kit Pang Szeto