Tata Consultancy Services Limited

Rekha Singhal

Manoj Karunakaran Nambiar

Shruti Kunde

Dheeraj Chahal

Mayank Mishra

Gautam Shroff

Chetan Dnyandeo Phalak

Lovekesh Vig

Manoj Karunakara Nambiar

Praveen Kumar Singh

Amit Sangroya

Rupinder Singh Virk

Sanket Kadarkar

Ashwin Krishnan

Chinmay Narendra Mahajan

Archisman Bhowmick

Manju Ramesh

Sharod Roy Choudhury

Siddharth Verma

Vartika Tewari

Rupinder Virk

High-performance deployment of DNN recommendation models heavily rely on embedding tables, and their performance bottleneck lies in the latency of embedding access. To optimize the deployment of RMs, the method and system is disclosed, which leverages heterogeneous memory types on FPGAs to improve the overall performance by maximizing the availability of frequently accessed data in faster memory. The system, using a optimizer dynamically allocates table partitions of the embedding tables based on history of input access history. A pre-optimizer block disclosed determines whether smaller tables should be partitioned or placed entirely in smaller memories, improving overall efficiency. The performance of RM is improved with improvement in average embedding fetch latency and effectively inference latency via modified Round Trip computation.

Pre-optimizer and optimizer based framework for optimal deployment of embedding tables across heterogeneous memory architecture

Cloud and Fog computing are complementary technologies used for complex Internet of Things (IoT) based deployment of applications. With an increase in the number of internet-connected devices, the volume of data generated and processed at higher speeds has increased substantially. Serving a large amount of data and workloads for predictive decisions in real-time using fog computing without Service-Level Objective (SLO) violation is a challenge. Present disclosure provides systems and method for inference management wherein a suitable execution workflow is automatically generated to execute machine learning (ML)/deep learning (DL) inference requests using fog with various type of instances (e.g., Function-as-a-Service (FaaS) instance, Machine Learning-as-a-service (MLaaS) instance, and the like) provided by cloud vendors/platforms. Generated workflow minimizes the cost of deployment as well as SLO violations.

Service-level objective (SLO) aware execution of concurrence inference requests on a fog-cloud network

Works in the literature fail to leverage embedding access patterns and memory units' access/storage capabilities, which when combined can yield high-speed heterogeneous systems by dynamically re-organizing embedding tables partitions across hardware during inference. A method and system for optimal deployment of embeddings tables across heterogeneous memory architecture for high-speed recommendations inference is disclosed, which dynamically partitions and organizes embedding tables across fast memory architectures to reduce access time. Partitions are chosen to take advantage of the past access patterns of those tables to ensure that frequently accessed data is available in the fast memory most of the time. Partition and replication is used to co-optimize memory access time and resources. Dynamic organization of embedding tables changes location of embedding, hence needs an efficient mechanism to track if a required embedding is present in the fast memory with its current address for faster look-up, which is performed using spline-based learned index.

Optimal deployment of embeddings tables across heterogeneous memory architecture for high-speed recommendations inference

The present disclosure provides a scalable acceleration of data processing in Machine Learning pipeline which is unavailable in conventional methods. Initially, the system receives a dataset and a data processing code. A plurality of sample datasets are obtained based on the received dataset using a sampling technique. A plurality of performance parameters corresponding to each of the plurality of sample datasets are obtained based on the data processing code using a profiling technique. A plurality of scalable performance parameters corresponding to each of a plurality of larger datasets are predicted based on the plurality of performance parameters and the data processing code using a curve fitting technique. Simultaneously, a plurality of anti-patterns are located in the data processing code using a pattern matching technique. Finally, an accelerated code is recommended based on the plurality of anti-patterns and the predicted plurality of scalable performance parameters using an accelerated code recommendation technique.

Method and system for scalable acceleration of data processing pipeline

This disclosure relates generally to configuring/building of applications. Typically, a deep learning (DL) application having multiple models composed and interspersed with corresponding transformation functions has no mechanism of efficient deployment on underlying system resources. The disclosed system accelerates the development of application to compose multiple models where each model could be a primitive model or a composite model itself. In an embodiment, the disclosed system optimally deploys a composable model application and transformation functions on underlying resources using performance prediction models, thereby accelerating the development and deployment of the application.

Efficient deployment of machine learning and deep learning model's pipeline for serving service level agreement

The disclosure generally relates to system architectures, and, more particularly, to a method and system for system architecture recommendation. In existing scenario, a solution architect often gets minimum details about requirements, hence struggles to design a system architecture that matches the requirements. The method and system disclosed herein are to provide system recommendation in response to requirements provided as input to the system. The system generates an acyclic dependency graph based on parameters and values extracted from an obtained user input. The system then identifies a reference architectures that matches the requirements, and further selects components that match the architecture requirements. The system further selects technologies considering inter-operability of the technologies. Further, the system generates architecture recommendations for the user, based on the selected components, and technologies. The system can collect user feedback for to the recommendation provided, and can generate rankings to improve future recommendations.

Method and system for system architecture technology recommendation

Business to Consumer (B2C) systems face a challenge of engaging users since offers are created using static rules generated using clustering on large transactional data generated over a period of time. Moreover, the offer creation and assignment engine is disjoint to the transactional system which led to significant gap between history used to create offers and current activity of users. Systems and methods of the present disclosure provide a meta-model based configurable auto-tunable recommendation model generated by ensembling optimized machine learning and deep learning models to predict a user's likelihood to take an offer and deployed in real time. Furthermore, the offer given to the user is based on a current context derived from the user's recent behavior that makes the offer relevant and increases probability of conversion of the offer to a sale. The system achieves low recommendation latency and scalable high throughput by virtue of the architecture used.

Systems and methods for real time configurable recommendation using user data

This disclosure relates generally to accelerating development and deployment of enterprise applications where the applications involve both data driven and task driven components in data driven enterprise information technology (IT) systems. The disclosed system is capable of determining components of the application that may be task-driven and/or those components which may be data-driven using inputs such as business use case, data sources and requirements specifications. The system is capable of determining the components that may be developed using task-driven and data-drive paradigms and enables migration of components from the task driven paradigm to the data driven paradigm. Also, the system trains a reinforcement learning (RL) model for facilitating migration of the identified components from the task driven paradigm to the data driven paradigm. The system is further capable of integrating the migrated and existing components to accelerate development and deployment an integrated IT application.

Accelerating development and deployment of enterprise applications in data driven enterprise IT systems

A system and method for estimating execution time of an application with Spark&#x2122; platform in a production environment. The application on Spark&#x2122; platform is executed as a sequence of Spark jobs. Each Spark job is executed as a directed acyclic graph (DAG) consisting of stages. Each stage has multiple executors running in parallel and the each executor has set of concurrent tasks. Each executor spawns multiple threads, one for each task. All jobs in the same executor share the same JVM memory. The execution time for each Spark job is predicted as summation of the estimated execution time of all its stages. The execution time constitutes scheduler delay, serialization time, de-serialization time, and JVM overheads. The JVM time estimation depends on type of computation hardware system and number of threads.

System and method for predicting application performance for large data size on big data cluster

Systems and methods for generating performance prediction model and estimating execution time for applications is provided. The system executes synthetic benchmarks for a first dataset on a first cluster. Each synthetic benchmark includes a MapReduce (MR) job. The system further extracts sensitive parameters for each sub-phase of the MR job, generates a linear regression prediction model for each sub-phase to obtain one or more linear regression prediction models, based on which the system further generates a performance prediction model to be utilized for predicting, using the sensitive parameters, a Hive query execution time of a Directed Acyclic Graph (DAG) of one or more MR jobs executed on a second dataset on a second cluster, wherein the first cluster that includes the first dataset is smaller compared to the second cluster that includes the second dataset.

Systems and methods for generating performance prediction model and estimating execution time for applications

System(s) and method(s) for predicting effect of database cache on query elapsed response time during an application development stage are disclosed. Query executed on a database is classified and a query type is obtained. Database statistics are collected to further determine one or more cache miss factors with respect to the query type. One or more time components are calculated due to the one or more cache miss factors with respect to the query type. The one or more time components are used to predict the query elapsed response time for varying size of the database.

System and method predicting effect of cache on query elapsed response time during application development stage

Disclosed is a system for predicting an Elapsed Response Time (ERT) of a query to be executed in a production database. The system includes a database emulator module for creating an emulated database based on empty schema derived by extrapolating statistical information from the development environment. The system also includes an execution plan generation module for generating a query execution plan of the query. The system further includes a query classifying module for classifying each sub-query based on the operational parameters into at least one of a full table scan, a primary index scan, a fast index scan and a non-unique index scan. The system also includes a time determination module for executing each sub-query on the emulated database to determine an Input Output (IO) access time and an operator time. Moreover, the system includes an ERT prediction module for predicting the ERT of the query to be executed in the production database based on the IO access time, the operator time, and a network time.

Method and system for efficient performance prediction of structured query for big data

A system, method and computer program product to predict elapsed response time of a query during an application stage is disclosed. The query is executed over a database and is classified based on a mode of access of the database by the query. The database is emulated to obtain an emulated database. After classification, an access pattern of the database by the query along with one or more parameters is determined. By applying a classification specific technique over the access pattern and the one or more parameters, an input-access time of the query on the emulated database is then calculated to determine the elapsed response time of the query.

Growing community of inventors

Mumbai, India

Rekha Singhal