Waymo LLC

Zhinan Xu

Chen Wu

Jens-Steffen Ralf Gutmann

Ryan M Geiss

Lucian Ion

Alper Ayvaci

Tiffany L Chen

Carl Warren Craddock

Ajay Chandra Venkata Gummalla

Pieter Kapsenberg

Xiaoying He

Lo Po Tsui

Maya Kabkab

Sean Rafferty

Changchang Wu

Woojong Koh

Justin Thorsen

David Ian Ferguson

David Harrison Silver

David Ian Franklin Ferguson

Andreas Wendel

Brandon Douglas Luders

Bradley Dodson

A system is provided that includes an image sensor coupled to a vehicle, and control circuitry configured to perform operations including receiving, from the image sensor, an input stream comprising high dynamic range (HDR) image data associated with an environment of the vehicle, and processing the input stream at the vehicle by applying a global tone mapping, followed by offline image processing that can include applying a local tone mapping to the globally tone mapped images of the same input stream.

Systems and methods for a tone mapper solution for an autonomous driving system

Aspects of the disclosure provide for automatically generating labels for sensor data. For instance, first sensor data, for a vehicle may be identified. This first sensor data may have been captured by a first sensor of the vehicle at a first location during a first point in time and may be associated with a first label for an object. Second sensor data for the vehicle may be identified. The second sensor data may have been captured by a second sensor of the vehicle at a second location at a second point in time outside of the first point in time. The second location is different from the first location. A determination may be made as to whether the object is a static object. Based on the determination that the object is a static object, the first label may be used to automatically generate a second label for the second sensor data.

Automatic labeling of objects in sensor data

The technology relates to predicting that an object is going to enter into a trajectory of a vehicle. This may include receiving sensor data identifying a first location of the object in an environment of the vehicle at a first point in time and receiving sensor data identifying a second location of the object in the environment at a second point in time. In addition, a boundary of the trajectory is determined by defining at least a two-dimensional area through which the vehicle is expected to travel in the future. A first distance between the boundary and the first location and a second distance between the trajectory and the second location are determined. The first distance and the second distance are used to determine that the object is going to enter into the trajectory at a future point in time.

Predicting trajectory intersection by another road user

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detecting foreign objects using a surfel map. One of the methods includes receiving sensor data representing reflection characteristics of electromagnetic waves in an operating environment of an autonomous vehicle. A surfel map representing a portion of the operating environment of the autonomous vehicle is used to determine that the reflection data includes one or more mismatched reflection characteristics for a particular surfel in the surfel map. In response, a foreign object is detected at a location corresponding to the particular surfel having the one or more mismatched reflection characteristics.

Foreign object detection with surfel maps

Aspects of the disclosure relate to determining a sign type of an unfamiliar sign. The system may include one or more processors. The one or more processors may be configured to receive an image and identify image data corresponding to a traffic sign in the image. The image data corresponding to the traffic sign may be input in a sign type model. The processors may determine that the sign type model was unable to identify a type of the traffic sign and determine one or more attributes of the traffic sign. The one or more attributes of the traffic sign may be compared to known attributes of other traffic signs and based on this comparison, a sign type of the traffic sign may be determined. The vehicle may be controlled in an autonomous driving mode based on the sign type of the traffic sign.

Detecting unfamiliar signs

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a depth map of a scene from a camera image using a neural network. One of the methods includes obtaining an image captured by a first sensor. A neural network processes the image to generate a respective score for each of a plurality of locations in the image. Known depth data specifying respective known depth values for some of the locations in the image is obtained. A depth output is generated that assigns a depth value to some of the locations in the image, including determining whether the score for a location exceeds a threshold; and when the score exceeds the threshold and the known depth value is available for the location, assigning the known depth value for the location to the location in the depth output.

Generating depth from camera images and known depth data using neural networks

Aspects of the disclosure provide for automatically generating labels for sensor data. For instance, first sensor data for a vehicle may be identified. This first sensor data may have been captured by a first sensor of the vehicle at a first location during a first point in time and may be associated with a first label for an object. Second sensor data for the vehicle may be identified. The second sensor data may have been captured by a second sensor of the vehicle at a second location at a second point in time outside of the first point in time. The second location is different from the first location. A determination may be made as to whether the object is a static object. Based on the determination that the object is a static object, the first label may be used to automatically generate a second label for the second sensor data.

Aspects of the disclosure relate to maneuvering a vehicle. As an example, sensor information identifying a set of objects as well as a set of characteristics for each object of the set of objects is received from a perception system of a vehicle. The set of objects is filtered to remove objects corresponding to vehicles, bicycles, and pedestrians. An object within an expected future path of the vehicle is selected from the filtered set of objects. The object is classified as drivable or not drivable based on the set of characteristics. Drivable indicates that the vehicle can drive over the object without causing damage to the vehicle. The vehicle is maneuvered based on the classification such that when the object is classified as drivable, maneuvering the vehicle includes driving the vehicle over the object by not altering the expected future path of the vehicle.

Growing community of inventors

Mountain View, CA, United States of America

Zhinan Xu