Google Inc.

Waymo LLC

Chen Wu

Debargha Mukherjee

Meng Wang

Sang-Uok Kum

Jonathon Shlens

Alper Ayvaci

Zhinan Xu

Abhijit S Ogale

David Harrison Silver

Congcong Li

Pankaj Chaudhari

Carl Kershaw

Henrik Kretzschmar

Victoria Dean

Jonathan J Huang

Andreas Wendel

Zhifeng Chen

Vijay Vasudevan

Tiffany L Chen

Ruichi Yu

Vijnan Shastri

Junhua Mao

Carl Warren Craddock

Jiquan Ngiam

Ouais Alsharif

Thierry Foucu

Wei Han

Lo Po Tsui

Yu-Han Chen

Sam Kvaalen

Maya Kabkab

Sean Rafferty

Benjamin James Caine

Brandon Chauloon Yang

Christoph Sprunk

Zhengdong Zhang

Changchang Wu

Noha Waheed Ahmed Radwan

Justin Thorsen

Woojong Koh

Yuchen Xie

Aspects of the disclosure provide for automatically generating labels for sensor data. For instance, first sensor data, for a vehicle may be identified. This first sensor data may have been captured by a first sensor of the vehicle at a first location during a first point in time and may be associated with a first label for an object. Second sensor data for the vehicle may be identified. The second sensor data may have been captured by a second sensor of the vehicle at a second location at a second point in time outside of the first point in time. The second location is different from the first location. A determination may be made as to whether the object is a static object. Based on the determination that the object is a static object, the first label may be used to automatically generate a second label for the second sensor data.

Automatic labeling of objects in sensor data

The disclosure relates to assessing operation of a camera. In one instance, a volume of space corresponding to a first vehicle in an environment of a second vehicle may be identified using sensor data generated by a LIDAR system of the second vehicle. An image captured by a camera of the second vehicle may be identified. The camera may have an overlapping field of view of the LIDAR system at a time when the sensor data was generated. An area of the image corresponding to the volume of space may be identified and processed in order to identify a vehicle light. The operation of the camera may be assessed based on the processing.

Camera assessment techniques for autonomous vehicles

Aspects of the disclosure relate to determining a sign type of an unfamiliar sign. The system may include one or more processors. The one or more processors may be configured to receive an image and identify image data corresponding to a traffic sign in the image. The image data corresponding to the traffic sign may be input in a sign type model. The processors may determine that the sign type model was unable to identify a type of the traffic sign and determine one or more attributes of the traffic sign. The one or more attributes of the traffic sign may be compared to known attributes of other traffic signs and based on this comparison, a sign type of the traffic sign may be determined. The vehicle may be controlled in an autonomous driving mode based on the sign type of the traffic sign.

Detecting unfamiliar signs

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing data generated by a sensing system that rotationally senses an environment. In one aspect, a method comprises partitioning a predetermined period of time into a plurality of sub-periods, wherein the predetermined period of time is a period of time for which data generated by the sensing system constitutes a complete rotational sensing of the environment; for each sub-period: receiving current data generated by the sensing system during the sub-period and characterizing a respective partial scene of the environment; processing the current data using an object detection neural network to generate a current object detection output that is specific to the respective partial scene of the environment.

Streaming object detection within sensor data

Aspects of the disclosure relate to training and using a phrase recognition model to identify phrases in images. As an example, a selected phrase list may include a plurality of phrases is received. Each phrase of the plurality of phrases includes text. An initial plurality of images may be received. A training image set may be selected from the initial plurality of images by identifying the phrase-containing images that include one or more phrases from the selected phrase list. Each given phrase-containing image of the training image set may be labeled with information identifying the one or more phrases from the selected phrase list included in the given phrase-containing images. The model may be trained based on the training image set such that the model is configured to, in response to receiving an input image, output data indicating whether a phrase of the plurality of phrases is included in the input image.

Phrase recognition model for autonomous vehicles

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating object interaction predictions using a neural network. One of the methods includes obtaining a sensor input derived from data generated by one or more sensors that characterizes a scene. The sensor input is provided to an object interaction neural network. The object interaction neural network is configured to process the sensor input to generate a plurality of object interaction outputs. Each respective object interaction output includes main object information and interacting object information. The respective object interaction outputs corresponding to the plurality of regions in the sensor input are received as output of the object interaction neural network.

Interacted object detection neural network

Aspects of the disclosure provide for automatically generating labels for sensor data. For instance, first sensor data for a vehicle may be identified. This first sensor data may have been captured by a first sensor of the vehicle at a first location during a first point in time and may be associated with a first label for an object. Second sensor data for the vehicle may be identified. The second sensor data may have been captured by a second sensor of the vehicle at a second location at a second point in time outside of the first point in time. The second location is different from the first location. A determination may be made as to whether the object is a static object. Based on the determination that the object is a static object, the first label may be used to automatically generate a second label for the second sensor data.

A two-dimensional image to be converted to a first three-dimensional image may be received. A second three-dimensional image that is visually similar to the two-dimensional image that is to be converted may be identified. A feature-to-depth mapping function may be computed for the first three-dimensional image by using an approximate depth map of the second three-dimensional image that is visually similar to the two-dimensional image that is to be converted. The feature-to-depth mapping function may be applied to a plurality of pixels of the two-dimensional image to determine a depth value for the plurality of pixels of the two-dimensional image. The first three-dimensional image may be generated based on the depth values for the plurality of pixels of the two-dimensional image.

Depth-map generation for an input image using an example approximate depth-map associated with an example similar image

Systems and methods for border detection on videos are disclosed herein. The system can include a refinement component that updates a variable as a function of a change in line number, wherein the change in line number is ascertained in response to a change in luminance values and/or chroma values associated with a border and a video area included in a frame associated with an original video. Further, the system also includes a trimming component that, as a function of the variable, crops the border from the frame that includes the video area of the resized original video.

Border detection in videos

Growing community of inventors

Mountain View, CA, United States of America

Chen Wu