Zoox, Inc.

Soroush Dean Khadem

Guillermo Duenas Arana

Till Kroeger

Francesco Papi

Derek Xiang Ma

David Burdick Berman

Daniel Miller

Richard Martin Murray

Anne-Claire Elisabeth Marie Le Henaff

Techniques for segmenting and classifying a representation of aggregated sensor data from a scene are discussed herein. Sensor data may be collected during multiple traversals of a same scene, and the sensor data may be filtered to remove portions of the sensor data not relevant to road network maps. In some examples, the filtered data may be aggregated and represented in voxels of a three-dimensional voxel space, from which an image representing a top-down view of the scene may be generated, though other views are also contemplated. Operations may include segmenting and/or classifying the image e.g., by a trained machine-learned model, to associate class labels indicative of map elements (e.g., driving lane, stop line, turn lane, and the like) with segments identified in the image. Additionally, techniques may create or update road network maps based on segmented and semantically labeled image(s) of various portions of an environment.

Semantic segmentation of aggregated sensor data

An evaluation computing system may implement techniques to validate a vehicle controller, such as based on an update thereto. The evaluation computing system may access data associated with an operation of the vehicle in an environment as controlled by the controller. The evaluation computing system may modify a portion of the data representative of a fault associated with the controller. The evaluation computing system may run a simulation utilizing modified log data to determine whether the controller detects and/or mitigates the fault within a threshold time. Based on a determination that the controller does not detect and/or mitigate the fault within the threshold time, the evaluation computing system may determine an error associated with the controller. Based on a determination that the controller detects and/or mitigates the fault within the threshold time, the evaluation computing system may validate the controller.

Fault detection in vehicle control systems

An evaluation computing system may implement techniques to validate a vehicle controller, such as based on a detection of a systematic fault. The evaluation computing system may access data (e.g., log data and/or map data) associated with an operation of the vehicle in an environment as controlled by the controller. The evaluation computing system may modify a portion of the map data representative of a simulated change associated with a portion of the environment. The evaluation computing system may run a simulation with a simulated environment, generated based on the modified map data, to determine whether the controller detects and/or mitigates the simulated change in a sufficient manner. Based on a determination of whether or not the controller detects and/or mitigates the simulated change in a sufficient manner, the evaluation computing system may determine an error associated with the controller or may validate the controller.

Systematic fault detection in vehicle control systems

Techniques for determining map data for localizing a vehicle in an environment are discussed herein. A vehicle can capture sensor data such as image data and can detect objects in the sensor data. Detected objects can include semantic object such as traffic lights, lane lines, street signs, poles, and the like. Rays based on a pose of the sensor and the detected objects can be determined. Intersections between the rays can be determined. The intersections can be utilized to determine candidate landmarks in the environment. The candidate landmarks can be utilized to determine landmarks and to eliminate other candidate landmarks. The map data can be determined based on the three-dimensional locations and utilized to localize vehicles.

Growing community of inventors

Boulder, CO, United States of America

Soroush Dean Khadem