Zoox, Inc.

Adam Pollack

Kratarth Goel

Derek Xiang Ma

Jesse Sol Levinson

Sarah Tariq

Mehran Ferdowsi

Scott M Purdy

Jonathan Tyler Dowdall

Ekaterina Hristova Taralova

Holly Ho

Justin Nordgreen

Oytun Ulutan

Bharadwaj Raghavan

William Harland Montgomery, Iv

Hang Ren

Techniques for detecting and labeling construction zones in an environment are disclosed. Two-dimensional images may be evaluated to identify pixels that may be associated with a construction zone and labeled accordingly. Labels for corresponding pixels in a separate image or non-construction zone labels for the same pixels may be compared to the construction zone pixels and an output image pixel label may be determined based on one or more criteria. An output image can be provided for vehicle control and for other operations, such as top-down segmentation and trajectory determination. Output images and related data may also be used to train a model to perform construction zone detection and labeling.

Systems and methods for construction zone detection

Techniques for adjusting vehicle models based on environmental conditions are discussed herein. The techniques may include receiving image data representing a portion of an environment in which a vehicle is operating and inputting the image data into a machine learned model. Additionally, data representing an environmental condition associated with the environment may be received from a sensor of the vehicle to detect changes in the environmental conditions such that one or more actions associated with the machine learned model or an output of the machine learned model may be performed. Some of the techniques may also include running multiple machine learned models or multiple configurations of a machine learned model in parallel and selecting different outputs of the machine learned model(s) to adjust for changes in the environmental conditions. For instance, individual outputs may be selected based on environmental conditions, confidence scores, thresholds, etc.

Adjusting vehicle models based on environmental conditions

Techniques for refining an image-based bounding box with depth data from a lidar sensor to represent an object in an environment of a vehicle are discussed herein. A vehicle computing device can implement a model to determine a three-dimensional bounding box based on image data. The vehicle computing device can implement a same of different model to update the three-dimensional bounding box based at least in part on depth information associated with lidar data. The techniques can also include training one or more machine learned models to determine three-dimensional bounding boxes usable by a vehicle to navigate in the environment.

Vision based object representation

Parallel vehicle model processing

Techniques for utilizing a depth completion algorithm to determine dense depth data are discussed are discussed herein. Two-dimensional image data representing an environment can be captured or otherwise received. Depth data representing the environment can be captured or otherwise received. The depth data can be projected into the image data and processed using the depth completion algorithm. The depth completion algorithm can be utilized to determine the dense depth values based on intensity values of pixels, and other depth values. A vehicle can be controlled based on the determined depth values.

Associating image data and depth data for depth determination

Techniques associated with generating or tuning parameters associated with long wave infrared sensor data to improve object detection associated with the captured images are discussed herein. The system may determine a region of interest associated with the sensor data and adjust or tune the parameters to improve detection(s) within the region of interest. Additionally, the system may adjust the parameters based on map data and/or environmental conditions, such as weather and temperature.

System for tuning parameters of a thermal sensor to improve object detection

System for tuning parameters of a thermal sensor based on a region of interest

Growing community of inventors

Foster City, CA, United States of America

Adam Pollack