Waymo LLC

Jennifer Taylor

David Harrison Silver

Benjamin T Ingram

Vaibhav Nangia

Angus Leigh

Samuel Liu

Ananya Misra

Frank Dachille

Brandon Douglas Luders

Aishwarya Parasuram

Sean Rafferty

Xin Liu

Bradley Dodson

Jiajing Wang

Sandro Alighiero Young

Enrique Fernandez Gonzalez

Dillon Cower

Nayun Xu

Lukasz Kulik

Karan Mahajan

Jayant Thatte

The described aspects and implementations support sound-based emergency vehicle detection, localization, and tracking for autonomous vehicle and driver-assist systems. The techniques include obtaining, using one or more audio detectors of a vehicle, a sound recording that includes a sound emitted by an emergency vehicle (EV). The techniques further include applying a sound localization (SL) model to the sound recording to obtain a SL output, which includes a first map of possible locations of the EV in a driving environment of the vehicle and can further include a second map of possible velocities of the EV. The techniques further include simulating, using the SL output, trajectories of simulated EV(s) in the driving environment of the vehicle, and causing, responsive to proximity of the simulated trajectories to a driving path of the vehicle, modification of the driving path of the vehicle.

Autonomous vehicle driving path selection in the presence of emergency vehicle sounds

The described aspects and implementations enable privacy-respecting detection, separation, and localization of sounds in vehicle environments. The techniques include obtaining, using audio detector(s) of a vehicle, a sound recording that includes a plurality of elemental sounds (ESs) in a driving environment of the vehicle, and processing, using a sound separation model, the sound recording to separate individual ESs of the plurality of ESs. The techniques further include identifying a content of individual ESs and causing a driving path of the vehicle to be modified in view of the identified content of the individual ESs. Further techniques include rendering speech imperceptibly by redacting temporal portions of the speech, using sound recognition models to identify and discard recordings of speech, and driving at speeds that exceed threshold speeds at which speech becomes imperceptible from noise masking.

Privacy-respecting detection and localization of sounds in autonomous driving applications

A system includes a first autonomous vehicle (AV) including a sensing system including a set of sensors, a memory storing instructions and a processing device operatively coupled to the memory, wherein the instructions, when executed by the processing device, cause the processing device to perform operations including observing, using data obtained from the sensing system, an event reflecting a scenario within a driving environment, generating a set of event observation data characterizing the event, and causing the set of event observation data to be shared with at least a second AV that is determined to be in a vicinity of the event after the event is observed.

Real-time multi-vehicle sensing and perception

The technology relates to detecting and responding to emergency vehicles. This may include using a plurality of microphones to detect a siren noise corresponding to an emergency vehicle and to estimate a bearing of the emergency vehicle. This estimated bearing is compared to map information to identify a portion of roadway on which the emergency vehicle is traveling. In addition, information identifying a set of objects in the vehicle's environment as well as characteristics of those objects is received from a perception system is used to determine whether one of the set of objects corresponds to the emergency vehicle. How to respond to the emergency vehicle is determined based on the estimated bearing and identified road segments and the determination of whether one of the set of objects corresponds to the emergency vehicle. This determined response is then used to control the vehicle in an autonomous driving mode.

Detecting and responding to sirens

The technology relates to detecting and responding to sounds for a vehicle having an autonomous driving mode. In one example, an audible signal corresponding to a sound received at one or more microphones of the vehicle may be received. Sensor data generated by a perception system of the vehicle identifying objects in an environment of the vehicle may be received. A type of sound may be determined by inputting the audible signal into a classifier. A set of additional signals may be determined based on the determined type of sound. The sensor data may be processed in order to identify one or more additional signals of the identified set of additional signals. The vehicle may be controlled in the autonomous driving mode in order to respond to the sound based on the one or more additional signals and the type of sound.

Detecting and responding to sounds for autonomous vehicles

Growing community of inventors

Palo Alto, CA, United States of America

Jennifer Taylor