Zoox, Inc.

Tod Cameron Semple

James Graham Dolan

Chonhyon Park

Maxwell Chandler Robinson

Chuang Wang

Leonardo Poubel Orenstein

Nathan David Shemonski

Priam Mukundan

Matthew Lotocki

Takahiro Baba

Techniques are discussed herein for executing simulations with parameterized object controllers to control smart agents, to evaluate the agent realism of the smart agents and to validate the efficacy of the simulations. A simulation system may analyze log data captured by a vehicle operating in a physical environment, to determine observable and non-observable behavior characteristics of the agents in the environment. The simulation system may execute simulations using object controllers to control simulated objects (e.g., 'smart agents') based on parameters associated with scenarios, object types, and/or scenario locations. Log data associated with smart agent behaviors may be aggregated and compared to the behavior characteristics of agents in physical environments, to determine metrics for agent realism and simulation efficacy. Based on such metrics, simulation results may be validated and/or object controller parameters may be modified.

Parameterized object controllers in driving simulations

Techniques are discussed herein for executing simulations including multiple simulated agents that can interact with each other with a reaction delay. At a first time point, a first trajectory of the first simulated agent associated with a first time point in the driving simulation can be determined, and stored in a first buffer component. At a second time point in the driving simulation after the first time point, the first trajectory of the first simulated agent associated with the first time point can be retrieved. A time period between the first time point and the second time point is a reaction delay. A second trajectory for controlling the second simulated agent can be determined based at least in part on the first trajectory of the first simulated agent associated with the first time point. The second simulated agent is controlled to move along the second trajectory.

Driving simulations with simulated agent interactions

Techniques associated with categorizing simulation instances associated with simulation testing comprise categorizing simulation instances that produce unique or otherwise interesting results for further testing, review, and validation. In some cases, the system may be configured to determine low period of resource consumption and to schedule simulation testing during those periods of time.

System for identifying simulation scenarios

Techniques for determining a modified lateral motion boundary of a vehicle in an environment are discussed herein. Techniques may include determining a lateral motion boundary for a vehicle in an environment, based partly on data associated with the environment and/or the vehicle, where representations of objects in the environment may be projected onto the lateral motion boundary. The lateral motion boundary may be modified based on such projections to determine a modified lateral motion boundary. The modified lateral motion boundary may include an open lateral boundary, configured by closing off portions of the lateral motion boundary corresponding to the projections, allowing a vehicle to move laterally to traverse the environment and avoid the objects.

Vehicle lateral avoidance

Techniques are discussed herein for executing log-based driving simulations to evaluate the performance and functionalities of vehicle control systems. A simulation system may execute a log-based driving simulation including playback agents whose behavior is based on the log data captured by a vehicle operating in an environment. The simulation system may determine interactions associated with the playback agents, and may convert the playback agents to smart agents during the driving simulation. During a driving simulation, playback agents that have been converted to smart agents may interact with additional playback agents, causing a cascading effect of additional conversions. Converting playback agents to smart agents may include initiating a planning component to control the smart agent, which may be based on determinations of a destination and/or driving attributes based on the playback agent.

Agent conversions in driving simulations

Driving simulations may be generated based on driving log data and used to validate autonomous vehicle safety systems. A driving simulation system may receive driving log data from autonomous vehicles and determine events which caused a vehicle to diverge from a planned trajectory. Driving simulations may be generated based on the driving log data, and executed using a simulated vehicle that follows the initial planned trajectory of the autonomous vehicle. The driving simulations may be analyzed to detect potential collisions and near misses, and to determine success conditions for the behaviors of vehicle safety systems. Driving simulations and associated success conditions may be aggregated and used in validation suites for vehicle controllers and/or vehicle safety systems, thereby providing more comprehensive and robust autonomous vehicle validation based on real-world driving scenarios.

Autonomous vehicle safety system validation

A system for generating simulated LiDAR data may include a depth offset generator and an intensity value generator. The depth offset generator may be configured to receive environment data including depth information, e.g., a depth map, the environment. The depth offset generator may determine an optical flow field from the depth information and estimate depths for positions to simulate LiDAR sensor data. field. The depth offset generator can also generate timestamp information based on attributes of the simulated LiDAR sensor, and determine the estimated depths using the timestamp information. The intensity value generator may be configured to determine an intensity for pixels based on physical attributes associated with those pixels. The simulated LiDAR data may be used in simulations run on autonomous vehicle control systems.

Growing community of inventors

Woodside, CA, United States of America

Tod Cameron Semple