Closed loop replay-based simulations

Abstract

Techniques for simulating environments are discussed herein. For example, a simulated autonomous vehicle may have simulated sensors to generate simulated sensor data based on log data. Using the log data, a simulated environment for testing a simulated vehicle may be generated. The simulated environment may represent the environment in which the real-world vehicle was operating. The techniques may further include generating simulated sensor data by the simulated sensors and sending the simulated sensor data to an autonomous vehicle controller where a trajectory of the simulated autonomous vehicle may be determined based on the simulated sensor data. In this way, an autonomous vehicle controller need not rely on the position of the vehicle in the original log data, and instead accurate simulated sensor data can be generated to determine accurate occlusions, amongst other things.