Training action prediction machine-learning models for video games with healed data

Abstract

This specification provides a computer-implemented method, the method comprising obtaining a machine-learning model. The machine-learning model is being trained with expert data comprising a plurality of training examples. Each training example comprises: (i) game state data representing a state of a video game environment, and (ii) scored action data representing an action and a score for that action if performed by a video game entity of the video game environment subsequent to the state of the video game environment. An action is performed by the video game entity based on a prediction for the action generated by the machine-learning model. The method further comprises determining whether the action performed by the video game entity was optimal. In response to determining that the action performed by the video game entity was suboptimal, a healed training example is generated. The healed training example comprises: (i) the state of the instance of the video game environment, and (ii) healed scored action data indicative that the action performed by the video game entity was suboptimal. The machine-learning model is updated based on the healed training example.