Sony Group Corporation

Google Inc.

Sony Interactive Entertainment Inc.

Sony Corporation of America

Piyush Khandelwal

Peter R Wurman

James MacGlashan

Rory Douglas

Dion Whitehead

Leon Barrett

Evan Jarman Fisher

Fabrizio Santini

Justin V Beltran

Todd Hester

Samuel Barrett

Kaushik Subramanian

Rabih Abdul Ahad

Bandaly Azzam

Leilani Gilpin

Houmehr Aghabozorgi

Thomas Walsh

Reinforcement learning methods can use actor-critic networks where (1) additional laboratory-only state information is used to train a policy that much act without this additional laboratory-only information in a production setting; and (2) complex resource-demanding policies are distilled into a less-demanding policy that can be more easily run at production with limited computational resources. The production actor network can be optimized using a frozen version of a large critic network, previously trained with a large actor network. Aspects of these methods can leverage actor-critic methods in which the critic network models the action value function, as opposed to the state value function.

Training actor-critic algorithms in laboratory settings

A real-time temporal convolution network (RT-TCN) algorithm reuses the output of prior convolution operations in all layers of the network to minimize the computational requirements and memory footprint of a TCN during real-time evaluation. Further, a TCN trained via the fixed-window view, where the TCN is trained using fixed time splices of the input time series, can be executed in real-time continually using RT-TCN.

Computing temporal convolution networks in real time

A user interface (UI), for analyzing model training runs, tracking and visualizing various aspects of machine learning experiments, can be used when training an artificial intelligent agent in, for example, a racing game environment. The UI can be web-based and can allow researchers to easily see the status of their experiments. The UI can include an experiment synchronized event viewer that can synchronizes visualizations, videos, and timeline/metrics graphs in the experiment. This viewer allows researchers to see how experiments unfold in great detail. The UI can further include experiment event annotations that can generate event annotations. These annotations can be displayed via the synchronized event viewer. The UI can be used to consider consolidated results across experiments and can further consider videos. For example, the UI can provide a reusable dashboard that can capture and compare metrics across multiple experiments.

User interface for operating artificial intelligence experiments

An artificial intelligent agent can act as a player in a video game, such as a racing video game. The game can be completely external to the agent and can run in real time. In this way, the training system is much more like a real world system. The consoles on which the game runs for training the agent are provided in a cloud computing environment. The agents and the trainers can run on other computing devices in the cloud, where the system can choose the trainers and agent compute based on proximity to console, for example. Users can choose the game they want to run and submit code which can be built and deployed to the cloud system. A resource management service can monitor game console resources between human users and research usage and identify experiments for suspension to ensure enough game consoles for human users.

Methods for controlling use of computing resources, such as virtual game consoles

In an embodiment, an electronic device may include a processor that may iteratively simulate candidate control trajectories using upper confidence bound for trees (UCT) to control an environmental control system (e.g., an HVAC system). Each candidate control trajectory may be simulated by selecting a control action at each of a plurality of time steps over a period of time that has the highest upper bound on possible performance using values from previous simulations and predicting a temperature for a next time step of the plurality of time steps that results from applying the selected control action using a thermal model. The processor may determine a value of each candidate control trajectory using a cost function, update the value of each control action selected in each candidate control trajectory, and select a candidate control trajectory with the highest value using UCT to apply to control the environmental control system.

Predictively controlling an environmental control system

Growing community of inventors

Austin, TX, United States of America

Piyush Khandelwal