Adobe, Inc.

Subrata Mitra

Sunav Choudhary

Nikhil Sheoran

Tong Yu

Anup Bandigadi Rao

Sungchul Kim

Handong Zhao

Sopan Khosla

Ayush Chauhan

Vibhor Porwal

Ruiyi Zhang

Aashaka Dhaval Shah

Rui Wang

Mekala Rajasekhar Reddy

Ryan A Rossi

Shivakumar Vaithyanathan

Fan Du

Iftikhar Ahamath Burhanuddin

Tung Mai

Koyel Mukherjee

Sanket Vaibhav Mehta

Kanak Vivek Mahadik

Gautam Choudhary

Sapthotharan Krishnan Nair

Yeuk-Yin Chan

Anil Malkani

Thomas W Jacobs

Anjali Yadav

Sai Keerthana Karnam

Manav Ketan Doshi

Shubham Agarwal

Syam Manoj Kumar Paila

Pradeep Dogga

Shaddy Garg

Anuj Jitendra Diwan

Arpit Aggarwal

Prateek Jain

Piyush Kumar Maurya

Sheng Yang

Samir Khuller

Shanka Subhra Mondal

Neeraj Jagdish Dhake

Ravinder Nehra

Ghetia Siddharth

Ramanuja Narasimha Simha

Jatin Varshney

Vidit Bhatia

Gautam Pratap Kowshik

Vikas Maddukuri

Sameeksha Arora

Sergey N Kazarin

Laxmikant Mishra

Himanshu Panday

Raunak Shah

Vishal Babu Bhavani

Dhruv Joshi

Shivam Pravin Bhosale

Priyanshu Yadav

Arjun Kashettiwar

Ghazi Shazan Ahmad

Sanjay Sukumaran

Jashn Arora

Munish Gupta

Kishalay Raj

Yash Gadhia

Atharv Tyagi

The present disclosure relates to systems, methods, and non-transitory computer-readable media that implements a secure distributed data collaboration architecture for generating synthetic datasets. For example, the disclosed system sends a request to perform a data collaboration with a first dataset of a first local node and a second dataset of a second local node. The disclosed system receives intermediate feature maps from the local nodes that correspond with the datasets and generates a combined feature map. Further, the disclosed system generates a synthetic dataset from the combined feature map by utilizing a central generative model. Moreover, the synthetic dataset generated by the disclosed system is statistically representative of the first dataset and the second dataset.

Providing data collaboration via a distributed secure collaboration framework

Methods and systems are provided for using reinforcement learning to recommend data visualizations. In embodiments described herein, statistical features for each sample of corresponding samples of a dataset are determined by applying each sample of the dataset to a data visualization recommendation model. The computational cost of each of the statistical features for each of the samples is determined based via a regression model. Recommended statistical features are determined by sequentially applying each sample to a reinforcement learning model with a computational budget and with the corresponding computational costs of the statistical features of each sample. A data visualization is then displayed that is generated by applying the dataset and the recommended statistical features to the data visualization recommendation model.

Using reinforcement learning to recommend data visualizations

A dataset comprising tables is received. Embeddings are generated for column titles of a table. Based on the embeddings, similar tables are clustered. The tables are organized into smaller clusters based on statistical similarities. Similarity scores are calculated for tables within the same cluster. A relatedness graph is created based on the similarity scores; similar tables are represented by nodes connected by edges. If the similarity score for a pair of tables exceeds a threshold, a table is deleted.

Relating data in data lakes

Embodiments of the technology described herein describe a machine classifier capable of continually learning new classes through a continual few-shot learning approach. A natural language processing (NLP) machine classifier may initially be trained to identify a plurality of other classes through a conventional training process. In order to learn a new class, natural-language training data for a new class is generated. The training data for the new class may be few-shot training data. The training also uses synthetic training data that represents each of the plurality of other classes. The synthetic training data may be generated through a model inversion of the original classifier. The synthetic training data and the natural-language training data are used to retrain the NLP classifier to identify text in the plurality of other classes and the new class using.

Teaching a machine classifier to recognize a new class

Shared resource interference detection techniques are described. In an example, a resource detection module supports techniques to quantify levels of interference through use of working set sizes. The resource detection module selects working set sizes. The resource detection module then initiates execution of code that utilizes the shared resource based on the first working set size. The resource detection module detects a resource consumption amount based on the execution of the code. The resource detection module then determines whether the detected resource consumption amount corresponds to the defined resource consumption amount for the selected working set size.

Shared resource interference detection involving a virtual machine container

To retrieve information derived from a plurality of separately stored datasets, join structures are identified within the plurality of separately stored datasets. Join structures can include datasets joined by a central dataset, datasets joined by a single key, and datasets joined across a plurality of keys. Each of the join structures corresponds to a query processing schema that defines a sampling technique. When a join query is received as a SQL query, the join query identifies a portion of the plurality of separately stored datasets, from which a join structure is selected and a corresponding query processing schema is identified. The join query is reconstructed to form a reconstructed join query that comprises query processing schema instructions to derive the requested information using the sampling technique defined by the identified query processing schema.

Approximate query processing for join queries

Some techniques described herein relate to utilizing a machine-learning (ML) model to select respective samples for queries of a query sequence. In one example, a method includes receiving a query in a query sequence, where the query is directed toward a dataset. Samples are available as down-sampled versions of the dataset. The method further include applying an agent to select, for the query, a sample from among the samples of the dataset. The agent includes an ML model trained, such as via intent-based reinforcement learning, to select respective samples for queries. The query is then executed against the sample to output a response.

Intent-aware learning for automated sample selection in interactive data exploration

A resource control system is described that is configured to control scheduling of executable jobs by compute instances of a service provider system. In one example, the resource control system outputs a deployment user interface to obtain job information. Upon receipt of the job information, the resource control system communicates with a service provider system to obtain logs from compute instances implemented by the service provider system for the respective executable jobs. The resource control system uses data obtained from the logs to estimate utility indicating status of respective executable jobs and an amount of time to complete the executable jobs by respective compute instances. The resource control system then employs a machine-learning module to generate an action to be performed by compute instances for respective executable jobs.

Scheduling and control of executable jobs over compute instances

The technology described herein is directed to a self-learning application scheduler for improved scheduling distribution of resource requests, e.g., job and service scheduling requests or tasks derived therefrom, initiated by applications on a shared compute infrastructure. More specifically, the self-learning application scheduler includes a reinforcement learning agent that iteratively learns a scheduling policy to improve scheduling distribution of the resource requests on the shared compute infrastructure. In some implementations, the reinforcement learning agent learns inherent characteristics and patterns of the resource requests initiated by the applications and orchestrates placement or scheduling of the resource requests on the shared compute infrastructure to minimize resource contention and thereby improve application performance for better overall user-experience.

Self-learning scheduler for application orchestration on shared compute cluster

Embodiments relate to tenant-side detection and mitigation of performance degradation resulting from interference generated by a noisy neighbor in a distributed computing environment. A first machine-learning model such as a k-means nearest neighbor classifier is operated by a tenant to detect an anomaly with a computer system emulator resulting from a co-located noisy neighbor. A second machine-learning model such as a multi-class classifier is operated by the tenant to identify a contended resource associated with the anomaly. A corresponding trigger signal is generated and provided to trigger various mitigation responses, including an application/framework-specific mitigation strategy (e.g., triggered approximations in application/framework performance, best-efforts paths, run-time changes, etc.), load-balancing, scaling out, updates to a scheduler to avoid impacted nodes, and the like. In this manner, a tenant can detect, classify, and mitigate performance degradation resulting from a noisy neighbor.

Tenant-side detection, classification, and mitigation of noisy-neighbor-induced performance degradation

Techniques are provided for scheduling multiple jobs on one or more cloud computing instances, which provide the ability to select a job for execution from among a plurality of jobs, and to further select a designated instance from among a plurality of cloud computing instances for executing the selected job. The job and the designated instance are each selected based on a probability distribution that a cost of executing the job on the designated instance does not exceed the budget. The probability distribution is based on several factors including a cost of prior executions of other jobs on the designated instance and a utility function that represents a value associated with a progress of each job. By scheduling select jobs on discounted cloud computing instances, the aggregate utility of the jobs can be maximized or otherwise improved for a given budget.

Scheduling jobs on interruptible cloud computing instances

A digital environment includes multiple computing nodes and a scheduling system that assigns workloads to computing nodes. The scheduling system includes an equivalence-class-based resource usage prediction system that receives a workload request and predicts an equivalence class for that workload request based on resource usage over time by the workload request or metadata associated with the workload request. The scheduling system also includes a workload assignment system that assigns the workload request to one or more of the computing nodes based on the predicted equivalence class. The number of equivalence classes is small relative to the total number of workloads that are scheduled (as an example, 10 to 15 equivalence classes for a total number of workloads in the tens or hundreds of thousands).

System and method for training and selecting equivalence class prediction modules for resource usage prediction

A method performed by one or more processors that preserves a machine learning model comprises accessing model parameters associated with a machine learning model. The model parameters are determined responsive to training the machine learning model. The method comprises generating a plurality of model parameter sets, where each of the plurality of model parameter sets comprises a separate portion of the set of model parameters. The method comprises determining one or more parity sets comprising values calculated from the plurality of model parameter sets. The method comprises distributing the plurality of model parameter sets and the one or more parity sets among a plurality of computing devices, where each of the plurality of computing devices stores a model parameter set of the plurality of model parameter sets or a parity set of the one or more parity sets. The method comprises accessing, from the plurality of computing devices, a number of sets comprising model parameter sets and at least one parity set. The method comprises reconstructing the machine learning model from the number of sets accessed from the plurality of computing devices.

Managing machine learning model reconstruction

In some embodiments, a model training system trains a sample generation model configured to generate synthetic data entries for a dataset. The sample generation model includes a prior model for generating an estimated latent vector from a partially observed data entry, a proposal model for generating a latent vector from a data entry of the dataset and a mask corresponding to the partially observed data entry, and a generative model for generating the synthetic data entries from the latent vector and the partially observed data entry. The model training system trains the sample generation model to optimize an objective function that includes a first term determined using the synthetic data entries and a second term determined using the estimated latent vector and the latent vector. The trained sample generation model can be executed on a client computing device to service queries using the generated synthetic data entries.

Query-oriented approximate query processing based on machine learning techniques

Growing community of inventors

Karnataka, India

Subrata Mitra