Zoox, Inc.

Ryan Martin Cahoon

Gary Linscott

Jacob Lee Askeland

Adrien Treuille

James Graham Dolan

Andrew Lewis King

Rasoul Kabirzadeh

Shao-Han Tang

Alireza Ashayer

Christopher Yeates Brown

Ryder Rishel

Techniques for instantiating objects in a simulated environment based on log data are disclosed herein. Some of the techniques may include receiving log data representing an environment in which a real-world vehicle was operating. Using the log data, a simulated environment for testing a simulated vehicle may be generated. The simulated environment may represent the environment in which the real-world vehicle was operating. The techniques may further include determining a location of the simulated vehicle as it traverses the simulated environment. Based at least in part on the log data, a prior location of the real-world vehicle in the environment closest to the location of the simulated vehicle in the simulated environment may be determined. In this way, the simulated environment may be updated to include a simulated object representing an object in the environment that was perceived by the vehicle from the prior location.

Instantiating objects in a simulated environment based on log data

A controller may control input to, output from, and execution of a component of a data-processing pipeline via an interface. The interface facilitates replacing the component with a different and/or updated component and/or changing a type of controller that controls the component via the interface. For example, the different types of controllers may facilitate communication between components controlled by other controllers (and/or that aren't controlled by a controller), controllers that generate reproducibility data so that component behavior may be reproduced, controllers that reproduce component behavior, and/or controllers that tune performance of the data-processing pipeline, e.g., by varying input, output, and execution of respective component(s).

Executable component interface and controller

A controller may use a timing model to control and coordinate the execution of components within a data processing pipeline. The timing model may be generated based on log data collected during execution of the components, to model component latencies and pipeline behavior in a production environment. The controller may use the timing model to control the timing of the inputs to, the outputs from, and the execution of various interacting components in the data processing pipeline. The timing model may include component-specific and/or channel-specific timing data, and in various implementations may be based on inputs provided to the components, the timing and contents of outputs from the components, and/or the number of times particular code is executed within the components.

Modeling component latencies in data processing pipelines

This disclosure describes techniques for latency mitigation during inter-process communication based on pre-allocating and managing shared memory. During the startup phase (e.g., prior to run-time), processes of vehicle computing systems may request to engage in communication with other processes within the vehicle computing system. Based on the request, a memory manager may determine an amount of shared memory to pre-allocate. In such examples, the amount of shared memory may be determined based on a maximum number of messages that may be 'in-flight' at any single instance during run-time, and a maximum size of a message. The memory manager may pre-allocate shared memory consistent with the determined amount, and transmit a key to the requesting process. In such examples, the key may enable the process to access the pre-allocated shared memory. After the startup phase has ended, the process may read and/or write messages to and from the shared memory.

Latency mitigation for inter-process communication

A domain specific language for use in constructing simulations within real environments is described. In an example, a computing device associated with a vehicle can receive, from one or more sensors associated with the vehicle, sensor data associated with an environment within which the vehicle is positioned. In an example, the vehicle can be an autonomous vehicle. The computing device associated with the vehicle can receive simulated data associated with one or more primitives that are to be instantiated as a scenario in the environment. The computing device can merge the sensor data and the simulated data to generate aggregated data and determine a trajectory along which the vehicle is to drive based at least in part on the aggregated data. The computing device can determine instructions for executing the trajectory and can assess the performance of the vehicle based on how the vehicle responds to the scenario.

Onboard use of scenario description language

A domain specific language, or Scenario Description Language (SDL), can be used for quickly enumerating scenarios in a simulation for testing and validating interaction of an object (e.g., an autonomous vehicle) within an environment. Scenarios in a simulation are defined using one or more primitives. Primitives are used to define objects to be instantiated (such as body size, position, orientation, velocities, etc.) and/or actions to be performed by the objects in the simulation (such as wait for a period of time, goal positions, follow a particular object, etc.). The SDL enables simple creation of multiple scenarios by combining primitives combinatorially and in some examples, limiting which scenarios are created to those that correspond to combinations that provide meaningful information. Additionally, the SDL allows for instantiation to be agnostic of map features so that a particular scenario can be instantiated automatically over all possible positions within a map.

Scenario description language

A coordinated component interface control framework may deterministically reproduce behavior of a data processing pipeline. The framework may include a controller that controls input to, output from, and/or execution of a component of the pipeline. The framework may also tune performance of the pipeline and/or enable parallel processing of the pipeline, even across different machines, while preserving the ability to deterministically reproduce behavior of the pipeline.

Growing community of inventors

Mountain View, CA, United States of America

Ryan Martin Cahoon