Zoox, Inc.

Shaminda Subasingha

Venkata Subrahmanyam Chandra Sekhar Chebiyyam

Robert Nicholas Moor

Nam Gook Cho

Samantha Marie Ting

Yongzhe Chen

Samuel Holladay

Xuan Zhong

Jonathan Solheim

Wuyang Yu

Subhasis Das

Turhan Karadeniz

Andrew E Beller

Jacob Avi Harper

Mark Alan Bates

Kevin Mark Karol

George Dalman Nelson, Iii

David Donald Logan

Arvind Pattabhiraman

Seyed Parsa Beheshti

Venkata Subrahrnanyarn Chandra Sekhar Chebiyyam

Anoop Jatavallabha Vijayakumar

William Randolph Geddings, Jr

Lowell Ray Pickett

Arvind Shivu Govinday

Techniques for detecting impacts to a vehicle body are described herein. A vehicle may receive first audio data (associated with a sound) from a microphone located inside the vehicle. Further, the vehicle may receive second audio data (associated with the sound) from a microphone that is external to the vehicle. The vehicle can compare, for example, the volumes of the first and second audio to determine a difference. The vehicle can compare the difference to a threshold to determine whether the sound corresponds to an impact to the vehicle body. For example, if the difference is below the threshold (e.g., the volume levels of the first and second audio data are similar), the vehicle can classify the sound as corresponding to an impact to the vehicle. The vehicle can control the vehicle based on the sound being classified as an impact to the vehicle body.

Detecting vehicle impacts based on interior microphones

A machine-learned (ML) model for detecting that depth data (e.g., lidar data, radar data) comprises a split beam as opposed to a dual return caused by particulate matter, such as dust, steam, smoke, rain, etc. may classify split beam returns as forming a hollow shape or as forming a solid shape. The ML model may classify split returns forming hollow shapes as being associated with multiple solid surfaces, whereas split returns forming solid shapes may be classified as being associated with particulate matter. Determining a cluster, centroid, density, an image detection, and/or the like may additionally or alternatively be used as input to the ML model and/or to weight outputs of the ml model.

Machine-learned classifier for lidar split returns

Techniques for determining audio data for output by an autonomous vehicle are discussed herein. A system can determine audio data representing a frequency and a sound level for different object types in an environment. The audio data can be specific for a human or animal, and can be adjusted based on a noise level in the environment, weather, ambient light, or other criteria. A model may determine audio data for continuous output in a vicinity of a specific object or region in the environment. A sound level of the audio data may be adjusted over time base on a likelihood an intersection between a vehicle path and an object path.

Autonomous vehicle sound determination by a model

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can return unreliable pixels, e.g., in the case of over-exposure or saturation. In some examples, composite sensor data can be generated based on information from multiple exposures captured at different exposure times. In examples, pixels in a first exposure are matched to pixels in a second exposure to account for relative movement of the vehicle and the detected object in the composite sensor data.

Generating composite data using long and short time-of-flight exposures

A machine-learned (ML) model for determining, using at least depth data (e.g., lidar data, radar data), whether an object exists beyond particulate matter, such as dust, steam, smoke, rain, etc. from a sensor. Determining the likelihood may include determining a probability map that varies, distance beyond the particulate matter, reflectance of the object, transmit power, or the like.

Particulate matter-occluded object probability map for sensor returns

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Composite sensor data can be generated based on information from multiple exposures captured at different exposure times. In examples, a first depth determination technique is applied to a first subset of pixels and a second depth determination technique is applied to a second subset of the pixels. The first subset of pixels may correspond to pixels that are associated with a highly-reflective surface.

Localized depth disambiguation for time-of-flight data

Systems and techniques for detecting and locating low impact collisions, such as relatively low energy impact collisions involving an autonomous vehicle, are disclosed herein. Sensor data from audio sensors configured on a vehicle may be used to determine whether a threshold number of such sensors have detected sounds associated with a collision. Scores determined for the sensors based on audio energy and confidence values can be used to determine a location estimate for the detected collision.

Event detection and localization using audio

Techniques for cleaning and calibrating a microphone are discussed herein. For example, a computing device can implement a microphone component to operate a speaker adjacent to a microphone to cause removal of an obstruction (e.g., rain, mud, dirt, dust, snow, ice, animal droppings, etc.) on or near the microphone. The microphone component can also or instead cause the speaker to output a frequency for testing operation of the microphone. By implementing the cleaning and/or calibrating techniques described herein, foreign particle(s) of varying size and type can be dislodged from an area near the microphone to improve performance of the microphone.

Microphone cleaning and calibration

A vehicle computing system may implement techniques to emit warning sounds from a vehicle to notify other entities in an environment with the vehicle of the vehicle operation. A vehicle computing system may be configured to emit warning sounds based on a speed associated with the vehicle. The vehicle computing system may determine a planned vehicle trajectory and that a speed corresponding to the planned vehicle trajectory is associated with the warning sound emission. The vehicle computing system may determine a time that the vehicle will be at the speed and may emit the warning sound substantially at the time. In some examples, the vehicle computing system may emit a different sound prior to the time and may transition to the warning sound associated with the speed at the time. By warning other entities of the vehicle operation, the warning sound emission system may improve the safety of vehicle operation.

Growing community of inventors

San Ramon, CA, United States of America

Shaminda Subasingha