Google Inc.

Dale Eric Schuurmans

Mohammad Norouzi

Ofir Nachum

Kelvin Xu

Craig Edgar Boutilier

Tyler Lu

Boon Teik Ooi

DiJia Su

Delusional bias can occur in function approximation Q-learning. Techniques for training and/or using a value network to mitigate delusional bias is disclosed herein, where the value network can be used to generate action(s) for an agent (e.g., a robot agent, a software agent, etc.). In various implementations, delusional bias can be mitigated by using a soft-consistency penalty. Additionally or alternatively, delusional bias can be mitigated by using a search framework over multiple Q-functions.

Mitigating delusional bias in deep Q-learning for robotic and/or other agents

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a policy neural network used to select actions to be performed by a reinforcement learning agent interacting with an environment. In one aspect, a method includes obtaining path data defining a path through the environment traversed by the agent. A consistency error is determined for the path from a combined reward, first and last soft-max state values, and a path likelihood. A value update for the current values of the policy neural network parameters is determined from at least the consistency error. The value update is used to adjust the current values of the policy neural network parameters.

Training policy neural networks using path consistency learning

Growing community of inventors

Mountain View, CA, United States of America

Dale Eric Schuurmans