Tata Consultancy Services Limited

Easwara Naga Subramanian

Sanjay Purushottam Bhat

Avinash Achar

Venkatesh Sarangan

Yogesh Kumar Bichpuriya

Abhay Pratap Singh

Akshaya Natarajan

Amit Kalele

Anubhav Jain

Narendhar Gugulothu

Vignesh Lakshmanan Kangadharan Palaniradja

This disclosure relates to method and system for optimal policy learning and recommendation for distribution task using deep RL model, in applications where when the action space has a probability simplex structure. The method includes training a RL agent by defining a policy network for learning the optimal policy using a policy gradient (PG) method, where the policy network comprising an artificial neural network (ANN) with a set of outputs. A continuous action space having a continuous probability simplex structure is defined. The learning of the optimal policy is updated based on one of stochastic and deterministic PG. For stochastic PG, a Dirichlet distribution based stochastic policy parameterized by output of the ANN with an activation function at an output layer of the ANN is selected. For deterministic PG, a soft-max function is selected as activation function at the output layer of the ANN to maintain the probability simplex structure.

Optimal policy learning and recommendation for distribution task using deep reinforcement learning model

This disclosure relates generally to system and method for time series prediction using a sparse recurrent mixture density network (RMDN), such as sparse LSTM-MDN and a sparse ED-MDN, for accurate forecasting of a high variability time series. The disclosed sparse RMDN has the ability to handle high-dimensional input features, capture trend shifts and high variability present in the data, and provide a confidence estimate of the forecast. A high-dimensional time series data is passed through a feedforward layer, which performs dimensionality reduction in an unsupervised manner by inducing sparsity on weights of the feedforward layer. The resultant low-dimensional time series is fed through recurrent layers to capture temporal patterns. These recurrent layers also aid in learning latent representation of the input data. Thereafter, a mixture density network (MDN) is used to model the variability and trend shifts present in the input and it also estimates the confidence of the predictions.

Time series prediction with confidence estimates using sparse recurrent mixture density networks

Sum of bid quantities (across price bands) placed by generators in energy markets have been observed to be either constant OR varying over a few finite values. Several researches have used simulated data to investigate desired aspect. However, these approaches have not been accurate in prediction. Embodiments of the present disclosure identified two sets of generators which needed specialized methods for regression (i) generators whose total bid quantity (TBQ) was constant (ii) generators whose total bid quantity varied over a few finite values only. In first category, present disclosure used a softmax output based ANN regressor to capture constant total bid quantity nature of targets and a loss function while training to capture error most meaningfully. For second category, system predicts total bid quantity (TBQ) of a generator and then predicts to allocate TBQ predicted across the various price bands which is accomplished by the softmax regression for constant TBQs.

Neural network based prediction of competition behaviour in energy markets

In energy markets in which bidding process is used to sell energy, it is important that a mechanism for deciding bidding amount is in place. State of the art systems in this domain have the disadvantage that they rely on simulation data, and also they make certain assumptions, and both the factors can affect accuracy of results when the systems are deployed and are expected to handle practical scenarios. The disclosure herein generally relates to energy markets, and, more particularly, to a method and a system for Reinforcement Learning (RL) based model for generating bids. The system trains a RL agent using historical data with respect to competitor bids places and Market Clearing Prices (MCPs). The RL agent then processes real-time inputs and generates bidding recommendations.

Method and system for building reinforcement learning (RL) based model for generating bids

The disclosure generally relates to principal component analysis (PCA) computation and, more particularly, to concurrent PCA computation. In one embodiment, a plurality of concurrent PCA requests are received by a server. An input matrix for each of the concurrent PCA requests is computed using a general purpose-graphical processing unit (GP-GPU) by the server. Further, tridiagnolization on the input matrix is performed on each of the concurrent PCA requests by a general purpose-graphical processing unit (GP-GPU) in the server to generate a tridiagonal matrix for each of the concurrent PCA requests. Furthermore, a plurality of eigen values and corresponding eigen vectors are computed for the tridiagonal matrix of each of the concurrent PCA requests by the server and subsequently back transformation of the eigen values and the eigen vectors is performed by the server for each of the concurrent PCA requests to obtain associated principal components.

Growing community of inventors

Hyderabad Telangana, India

Easwara Naga Subramanian