Tata Consultancy Services Limited

Indian Institute of Technology Bombay

Manoj Karunakaran Nambiar

Rekha Singhal

Subhasri Duttagupta

Dheeraj Chahal

Nishant Kumar Agrawal

Ajay Kattepur

Dhaval Shah

Nupur Sumeet

Rupinder Singh Virk

Ashwin Krishnan

Himanshu Kumar

Mukund Kumar

Arpan Pal

Arijit Mukherjee

Srinivasa Rao Chalamala

Sounak Dey

Shruti Kunde

Mahesh Damodar Barve

Sunil Anant Puranik

Amit Kalele

Swapnil Shashikant Rodi

Harshad Khadilkar

Anubhav Jain

Ishtiyaque Shaikh

Piyush Manavar

Chetan Dnyandeo Phalak

Siddhartha Prakash Duttagupta

Barnali Basak

Ansuma Basumatary

Chinmay Narendra Mahajan

Deeksha Kashyap

Meripe Ajay Kumar

Kavya Borra

R Narayanan

Karan Rawat

Sharyu Vijay Mukhekar

Shyam Sundar Das

Ambuj Pandey

Rihab Abdulrazak

Subhashri Duttagupta

Soumitra Keshari Nayak

Sirshendu Arosh

Surya Prakash

Barve Mahesh Damodar

Manesh Barve

The disclosure generally relates to an FPGA-based online 3D bin packing. Online 3D bin packing is the process of packing boxes into larger bins-Long Distance Containers (LDCs) such that the space inside each LDC is used to the maximum extent. The use of deep reinforcement learning (Deep RL) for this process is effective and popular. However, since the existing processor-based implementations are limited by Von-Neumann architecture and take a long time to evaluate each alignment for a box, only a few potential alignments are considered, resulting in sub-optimal packing efficiency. This disclosure describes an architecture for bin packing which leverages pipelining and parallel processing on FPGA for faster and exhaustive evaluation of all alignments for each box resulting in increased efficiency. In addition, a suitable generic purpose processor is employed to train the neural network within the algorithm to make the disclosed techniques computationally light, faster and efficient.

Field programmable gate array (FPGA) based online 3D bin packing

State of the art techniques provide dedicated High-Level Synthesis (HLS) performance estimator tools that can give insights on performance bottlenecks, stall rate, stall cause etc., in HLS designs. These estimators often limit themselves to simple loop topologies and limited pragma use which makes them unreliable for large designs with complex datapaths. Embodiments herein provide a method and system for non-intrusive profiling for high-level synthesis HLS based applications. The method provides a cycle-accurate, fine-grained performance profiling framework that is non-intrusive and provides an end-to-end profile of the design. Such profiling tool can help the designer/DSE tool to quickly identify the performance bottlenecks and have a guided approach towards tuning it.

Method and system for non-intrusive profiling of high-level synthesis (HLS) based applications

High-performance deployment of DNN recommendation models heavily rely on embedding tables, and their performance bottleneck lies in the latency of embedding access. To optimize the deployment of RMs, the method and system is disclosed, which leverages heterogeneous memory types on FPGAs to improve the overall performance by maximizing the availability of frequently accessed data in faster memory. The system, using a optimizer dynamically allocates table partitions of the embedding tables based on history of input access history. A pre-optimizer block disclosed determines whether smaller tables should be partitioned or placed entirely in smaller memories, improving overall efficiency. The performance of RM is improved with improvement in average embedding fetch latency and effectively inference latency via modified Round Trip computation.

Pre-optimizer and optimizer based framework for optimal deployment of embedding tables across heterogeneous memory architecture

State of the art techniques used for document processing and particularly for handling processing of images for data extraction have the disadvantage that they have large computational load and memory footprint. The disclosure herein generally relates to text processing, and, more particularly, to a method and system for generating a data model for text extraction from documents. The system prunes a pretrained base model using a Lottery Ticket Hypothesis (LTH) algorithm, to generate a LTH pruned data model. The system further trims the LTH pruned data model to obtain a structured pruned data model, which involves discarding filters that have filter sparsity exceeding a threshold of filter sparsity. The structured pruned data model is then trained from a teacher model in a Knowledge Distillation algorithm, wherein a resultant data model obtained after training the structured pruned data model forms the data model for text detection.

Method and system for generating a data model for text extraction from documents

This disclosure relates generally to a method and a system for computing using a field programmable gate array (FPGA) neuromorphic architecture. Implementing energy efficient Artificial Intelligence (AI) applications at power constrained environment/devices is challenging due to huge energy consumption during both training and inferencing. The disclosure is a FPGA architecture based neuromorphic computing platform, the basic components include a plurality of neurons and memory. The FPGA neuromorphic architecture is parameterized, parallel and modular, thus enabling improved energy/inference and Latency-Throughput. Based on values of the plurality of features of the data set, the FPGA neuromorphic architecture is generated in a modular and parallel fashion. The output of the disclosed FPGA neuromorphic architecture is the plurality of output spikes from the neuron, which becomes the basis of inference for computing.

Field programmable gate array (FPGA) based neuromorphic computing architecture

Works in the literature fail to leverage embedding access patterns and memory units' access/storage capabilities, which when combined can yield high-speed heterogeneous systems by dynamically re-organizing embedding tables partitions across hardware during inference. A method and system for optimal deployment of embeddings tables across heterogeneous memory architecture for high-speed recommendations inference is disclosed, which dynamically partitions and organizes embedding tables across fast memory architectures to reduce access time. Partitions are chosen to take advantage of the past access patterns of those tables to ensure that frequently accessed data is available in the fast memory most of the time. Partition and replication is used to co-optimize memory access time and resources. Dynamic organization of embedding tables changes location of embedding, hence needs an efficient mechanism to track if a required embedding is present in the fast memory with its current address for faster look-up, which is performed using spline-based learned index.

Optimal deployment of embeddings tables across heterogeneous memory architecture for high-speed recommendations inference

The disclosure generally relates to system architectures, and, more particularly, to a method and system for system architecture recommendation. In existing scenario, a solution architect often gets minimum details about requirements, hence struggles to design a system architecture that matches the requirements. The method and system disclosed herein are to provide system recommendation in response to requirements provided as input to the system. The system generates an acyclic dependency graph based on parameters and values extracted from an obtained user input. The system then identifies a reference architectures that matches the requirements, and further selects components that match the architecture requirements. The system further selects technologies considering inter-operability of the technologies. Further, the system generates architecture recommendations for the user, based on the selected components, and technologies. The system can collect user feedback for to the recommendation provided, and can generate rankings to improve future recommendations.

Method and system for system architecture technology recommendation

The present disclosure provides systems and methods for performance evaluation of Input/Output (I/O) intensive enterprise applications. Representative workloads may be generated for enterprise applications using synthetic benchmarks that can be used across multiple platforms with different storage systems. I/O traces are captured for an application of interest at low concurrencies and features that affect performance significantly are extracted, fed to a synthetic benchmark and replayed on a target system thereby accurately creating the same behavior of the application. Statistical methods are used to extrapolate the extract features to predict performance at higher concurrency level without generating traces at those concurrency levels. The method does not require deploying the application or database on the target system since performance of system is dependent on access patterns instead of actual data. Identical access patterns are re-created using only replica of database files of the same size as in the real database.

Systems and methods for performance evaluation of input/output (I/O) intensive enterprise applications

This disclosure relates generally to methods and systems for providing exactly-once transaction semantics for fault tolerant FPGA based transaction systems. The systems comprise middleware components in a server as well as client end. The server comprises Hosts and FPGAs. The FPGAs control transaction execution (the application processing logic also resides in the FPGA) and provide fault tolerance with high performance by means of a modified TCP implementation. The Hosts buffer and persist transaction records for failure recovery and achieving exactly-once transaction semantics. The monitoring and fault detecting components are distributed across the FPGA's and Hosts. Exactly-once transaction semantics is implemented without sacrificing performance by switching between a high performance mode and a conservative mode depending on component failures. PCIE switches for connectivity between FPGAs and Hosts ensure FPGAs are available even if Hosts fail. When FPGA's provide higher processing elements and memory, the Hosts may be eliminated.

Exactly-once transaction semantics for fault tolerant FPGA based transaction systems

A method and system is provided for pre-deployment performance estimation of input-output intensive workloads. Particularly, the present application provides a method and system for predicting the performance of input-output intensive distributed enterprise application on multiple storage devices without deploying the application and the complete database in the target environment. The present method comprises of generating the input-output traces of an application on a source system with varying concurrencies; replaying the generated traces from the source system on a target system where application needs to be migrated; gathering performance data in the form of resource utilization, through-put and response time from the target system; extrapolating the data gathered from the target system in order to accurately predict the performance of multi-threaded input-output intensive applications in the target system for higher concurrencies.

Method and system for pre-deployment performance estimation of input-output intensive workloads

Performance prediction systems and method of an Internet of Things (IoT) platform and applications includes obtaining input(s) comprising one of (i) user requests and (ii) sensor observations from sensor(s); invoking Application Programming Interface (APIs) of the platform based on input(s); identifying open flow (OF) and closed flow (CF) requests of system(s) connected to the platform; identifying workload characteristics of the OF and CF requests to obtain segregated OF and segregated CF requests, and a combination of open and closed flow requests; executing performance tests with the APIs based on the workload characteristics; measuring resource utilization of the system(s) and computing service demands of resource(s) from measured utilization, and user requests processed by the platform per unit time; executing the performance tests with the invoked APIs based on volume of workload characteristics pertaining to the application(s); and predicting, using queuing network, performance of the application(s) for the volume of workload characteristics.

Systems and methods for predicting performance of applications on an internet of things (IoT) platform

Systems and methods for benchmark based cross platform service demand prediction includes generation of performance mimicking benchmarks that require only application level profiling and provide a representative value of service demand of an application under consideration on a production platform, thereby eliminating need for actually deploying the application under consideration on a production platform. The PMBs require only a representative estimate of service demand of the application under test and can be reused to represent multiple applications. The PMBs are generated based on a skeletal benchmark corresponding to the technology stack used by the application under test and an input file generated based on application profiling that provides pre-defined lower level method calls, data flow sequences between multi-tiers of the application under test and send and receive network calls made by the application under consideration.

Systems and methods for benchmark based cross platform service demand prediction

Systems and methods for service demand based performance prediction using a single workload is provided to eliminate need for load testing. The process involves identifying a range of concurrencies for the application under test; capturing a single workload pertaining to the application under test; and iteratively performing for the identified range of concurrencies: generating an array of one or more predefined CPU performance metrics based on the captured single workload; generating an array of service demands based on the captured single workload and the generated array of the one or more pre-defined CPU performance metrics; computing an array of throughput based on the generated array of service demands; and updating the generated array of the one or more pre-defined CPU performance metrics based on the computed array of throughput.

Service demand based performance prediction using a single workload

Systems and methods for service demand based performance prediction with varying workloads are provided. Chebyshev nodes serve as optimum number of load testing points to minimize polynomial interpolation error rates. Chebyshev nodes are identified for a pre-determined range of concurrencies in the application under test. An array of interpolated service demands at the Chebyshev nodes, when integrated with a modified multi-server Mean Value Analysis (MVA) technique provides superior throughput and response time predictions.

Systems and methods for service demand based performance prediction with varying workloads

The present disclosure discloses a method and system for non-uniform intensity mapping using a high performance enterprise computing system with enhanced precision cooling, enabling extended over-clocking and over-voltage operation. A Kalman filter embedded in the processor predicts and corrects the input data flux for real-time use by taking care of over-clocking and over-voltage.

Non-uniform intensity mapping using high performance enterprise computing system

Growing community of inventors

Mumbai, India

Manoj Karunakaran Nambiar