Zoox, Inc.

Qualcomm Incorporated

Panosense Inc.

Subasingha Shaminda Subasingha

Turhan Karadeniz

Vivek Rajendran

Venkata Subrahmanyam Chandra Sekhar Chebiyyam

Venkatraman Atti

Venkatesh Krishnan

Mehran Ferdowsi

Harrison Thomas Waschura

Robert Nicholas Moor

Nam Gook Cho

Ryan McMichael

Denis Nikitin

Joseph Patrick Warga

Jacob Avi Harper

Silas Kogure Wilkinson

Yongzhe Chen

Forrest Leighton Merrill

Jeremy Yi-Xiong Yang

Venkatraman S Atti

Michael Moshe Kohen

Samuel Holladay

Josh Alexander Jimenez

Riley Andrews

Jonathan Tyler Dowdall

George Dalman Nelson, Iii

Ravi Sankar Mahankali

Sarah Tariq

Daniel Jared Sinder

Chuang Wang

Stephane Pierre Villette

Joshua Kriser Cohen

Sreevatsan Bhaskaran

Sandip Sarkar

Marc Wimmershoff

Badeea Ferdaous Alferdaous Alazem

Samantha Marie Ting

Amanda Brown Prescott

Chandra Mouli Polisetty

Duminda Dewasurendra

Dilip Bethanabhotla

Aleksandr Oysgelt

Venkata Subrahmanyam Chebiyyam

Subhasis Das

Ravi Verma Gogna

Michael Carsten Bosse

Markus Jost

Koun Han

Andrew Lewis King

Mark Alan Bates

Kevin Mark Karol

Pravin Kumar Ramadas

Sandra Ruiz

Nathan Duarte

Anton Mario Bongio Karrman

Maxwell Ethan Mckenzie Yaron

Nathaniel John Villaume

Timothy John Leo Koenig

Marina Camille Josephs

Minh Nguyen

Valerie Bumbaca Randolph

Venkatraman Srinivasa Atti

Techniques for detecting and responding to an emergency vehicle are discussed. A vehicle computing system may determine that an emergency vehicle based on sensor data, such as audio and visual data. In some examples, the vehicle computing system may determine aggregate actions of objects (e.g., other vehicles yielding) proximate the vehicle based on the sensor data. In such examples, a determination that the emergency vehicle is operating may be based on the actions of the objects. The vehicle computing system may, in turn, identify a location to move out of a path of the emergency vehicle (e.g., yield) and may control the vehicle to the location. The vehicle computing system may determine that the emergency vehicle is no longer relevant to the vehicle and may control the vehicle along a route to a destination. Determining to yield and/or returning to a mission may be confirmed by a remote operator.

Emergency vehicle detection and response

Techniques for determining multipath interference associated with a surface in an environment based on sensor data are discussed herein. The sensor data may be captured and received from a sensor associated with a vehicle travelling through an environment. The sensor data may include depth data (which may be received as phase data) and intensity data associated with the environment. The depth data and the intensity data may be associated with various surfaces in the environment. Based on the relative changes between corresponding depth data and intensity data for a particular surface, multipath interference associated with either of the depth data or the intensity data may be determined.

Multipath interference detection

Techniques for determining a distance to an object in an environment based on sensor data are discussed herein. The sensor data can be captured by a sensor, and the sensor data can be based on laser pulses emitted from emitters being differently spaced apart from the sensor. The sensor data can be utilized to determine distance data associated with an object. A difference between the distance data can be used to determine whether the distance data is associated with a retroreflector. If the difference in distance data is above a threshold, a confidence level of sensor data can be adjusted or the sensor data can be discarded or omitted from subsequent processing.

Glare detection in sensor data

Techniques are discussed herein for analyzing radar data to determine that radar noise from one or more target detections potentially conceals additional objects near the target detection. Determining whether an object may be concealed can be based at least in part on a radar noise level based on a target detection, as well as distributions of radar cross sections and/or doppler data associated with particular object types. For a location near a target detection, a radar system may determine estimated noise levels, and compare the estimated noise levels to radar cross section probabilities associated with object types to determine the likelihood that an object of the object type could be concealed at the location. Based on the analysis, the system may determine a vehicle trajectory or otherwise may control a vehicle based on the likelihood that an object may be concealed at the location.

Radar data analysis and concealed object detection

Techniques for image data protection using cyclic redundancy checks are disclosed herein. Some of the techniques may include, at a processor, receiving image data that includes multiple lines of pixel data. The processor may also determine at least a first hash value representing a first line of pixel data of the multiple lines of pixel data and a second hash value representing a second line of pixel data of the multiple lines of pixel data. The processor may also send the image data to a computing device that is configured to determine, based at least in part on the first hash value and the second hash value, whether the image data is corrupt.

Image data protection using data integrity checks

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. The sensor may generate first image data at a first configuration and second image data at a second configuration. An estimated depth of an object may be determined from the first image data, and an actual depth of the object may be determined from the second image data, based on the estimated depth. In examples, the first and second configurations have different modulation frequencies such that a nominal maximum depth in the first configuration is greater than the nominal maximum depth in the second configuration.

Determining depth using multiple modulation frequencies

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can be impacted by glare. In some examples, an emitter system for illuminating a field of view can be dynamically altered to provide different illumination intensities at different regions in the field of view. For instance, locations at which a highly-reflective object or other object that is likely to cause glare may be illuminated at a lower illumination intensity, e.g., to reduce the likelihood of pixel saturation in measured data associated with the object.

Dynamic sensor illumination

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can include returns associated with highly reflective objects that cause glare. In some examples, a depth of a sensed surface is determined from the sensor data and additional pixels at the same depth are identified. The subset of pixels at the depth are filtered by comparing a measured intensity value to a threshold intensity value for the depth. Other threshold intensity values can be applied to subsets of pixels at different depths.

Depth dependent pixel filtering

Techniques for determining information associated with sounds detected in an environment based on audio data and map data or perception data are discussed herein. A vehicle can use map data and/or perception data to distinguish between multiple audio signals or sounds. A direct source of sound can be distinguished from a reflected source of sound by determining a direction of arrival of sounds and which objects the directions of arrival are associated with in the environment. A reflected sound can be received without receiving a direct sound. Based on the reflected sound and map data or perception data, characteristics of sound in an occluded region of the environment may be determined and used to control the vehicle.

Distinguishing between direct sounds and reflected sounds in an environment

This disclosure relates to systems and techniques for identifying collisions, such as relatively low energy impact collisions involving an autonomous vehicle. Sensor data from a first sensor modality in a first array may be used to determine a first estimated location of impact and second sensor data from a second sensor modality in a second array may be used to determine a second estimated location of impact. A low energy impact event may be configured when the first estimated location of impact corresponds to the second estimated location of impact.

Low-energy impact collision detection

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. The sensor may generate first image data at a first configuration and second image data at a second configuration. A disambiguated depth of a surface may be determined from the first image data and the second image data. If the disambiguated depth is greater than a nominal maximum depth of the sensor in the first configuration, an intensity of the surface may be determined from the first image data. If the intensity meets or exceeds a threshold intensity, the surface may be determined to be beyond the nominal maximum depth. If the intensity is less than the threshold intensity, an actual depth of the surface may be determined form the second image data as a distance less than the nominal maximum depth.

Determining pixels beyond nominal maximum sensor depth

Techniques for utilizing microphone or audio data to detect and responding to low velocity impacts to a system such as an autonomous vehicle. In some cases, the system may be equipped with a plurality of microphones that may be used to detect impacts that fail to register on the data captured by the vehicle's inertial measurement units and may go undetected by the vehicle's perception system and sensors. In one specific example, the perception system of the autonomous vehicle may identify a period of time in which a potential low velocity impact may occur. The autonomous vehicle may then utilize the microphone or audio data associated with the period of time to determine if an impact occurred.

System to detect impacts

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can return unreliable pixels, e.g., in the case of over-exposure or saturation. In some examples, multiple exposures captured at different exposure times can be used to determine an overall saturation value or metric representative of the sensor data. The saturation value may be used to control parameters of the sensor. For instance, the saturation value may be used to determine power control parameters for the sensor, e.g., to reduce over- and/or under-exposure.

Power control of sensors using multiple exposures

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can include saturated pixels, e.g., due to over-exposure, sensing highly-reflective objects, and/or excessive ambient light. In some examples, parameters associated with power of a time-of-flight sensor can be altered based on characteristics of the saturated pixels, as well as information about non-saturated pixels neighboring the saturated pixels. For example, the neighboring pixels may provide information about whether saturation is due to ambient light, e.g., sunlight, or due to emitted light from the sensor.

Active power control using neighboring pixels

A machine-learned (ML) model for detecting that depth data (e.g., lidar data, radar data) comprises a false positive attributable to particulate matter, such as dust, steam, smoke, rain, etc. The ML model may be trained based at least in part on simulated depth data generated by a fluid dynamics model and/or by collecting depth data during operation of a device (e.g., an autonomous vehicle. In some examples, an autonomous vehicle may identify depth data that may be associated with particulate matter based at least in part on an outlier region in a thermal image. For example, the outlier region may be associated with steam.

Identifying and/or removing false positive detections from LIDAR sensor output

Growing community of inventors

San Diego, CA, United States of America

Subasingha Shaminda Subasingha