Zoox, Inc.

Panosense Inc.

Ryan McMichael

Denis Nikitin

Joseph Patrick Warga

Adam Berger

Brian Pilnick

Riley Andrews

Subasingha Shaminda Subasingha

Robert Nicholas Moor

Christopher William Labadie

Anubhav Thakur

Derek Adams

Daniel Miller

Daniel Glenn Johnson

Austin In-Jei Yi

Peter Thomas Mitros

Turhan Karadeniz

Brian Alexander Pesch

Mehran Ferdowsi

Harrison Thomas Waschura

Silas Kogure Wilkinson

Sreevatsan Bhaskaran

Timothy David Kentley-Klay

Bertrand Robert Douillard

Christopher David Canady

Irene Perali

Maxwell Yaron

Jesse Sol Levinson

James William Vaisey Philbin

Robert J Chen

Jacob Lee Askeland

Haiming Wang

Till Kroeger

Yongzhe Chen

Samuel Holladay

Holly Ho

Arthur Benemann

Sandra Ruiz

Daniel Elliot Schabb

Mark R Rosekind

Alejandro Ruiz

Sasidhar Saladi

Greg Zanghi

Joshua Franz

Garen Khanoyan

Clark Pentico

Lee Tsung

Jon Robert Torrey

Techniques for determining a distance to an object in an environment based on sensor data are discussed herein. The sensor data can be captured by a sensor, and the sensor data can be based on laser pulses emitted from emitters being differently spaced apart from the sensor. The sensor data can be utilized to determine distance data associated with an object. A difference between the distance data can be used to determine whether the distance data is associated with a retroreflector. If the difference in distance data is above a threshold, a confidence level of sensor data can be adjusted or the sensor data can be discarded or omitted from subsequent processing.

Glare detection in sensor data

Systems and techniques for detecting feature(s) and object(s) partially or completely obscured by stray light effects in the visible light spectrum are discussed herein. Non-visible light can be emitted from emitters. Light from an environment including visible and non-visible light can be captured at a sensing system via a lens system. The sensing system can filter the captured light using a dual band-pass filter to direct certain non-visible and visible light spectra to a beam splitter that then directs the non-visible light to a non-visible light sensor and the visible light to a visible light sensor. The sensor data generated by the non-visible light sensor can then be used to identify and classify feature(s) and object(s) that may be obscured by stray light effects in the data generated by the visible light sensor.

Detecting dark objects in stray light halos

A LIDAR system emits laser bursts, wherein each burst has at least a pair of pulses. The pulses of each pair are spaced by a time interval having a variable duration to reduce effects of cross-talk. For example, certain embodiments may have multiple emitter/sensor channels that are used sequentially, and each channel may use a different duration for inter-pulse spacing to reduce the effects of cross-talk between channels. The durations may also be varied over time. The emitters and sensors are physically arranged in a two-dimensional array to achieve a relatively fine vertical pitch. The array has staggered rows that are packed using a hexagonal packing arrangement. The channels are used in a sequential order that is selected to maximize spacing between consecutively used channels, further reducing possibilities for inter-channel cross-talk.

Lidar system

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can be impacted by glare. In some examples, an emitter system for illuminating a field of view can be dynamically altered to provide different illumination intensities at different regions in the field of view. For instance, locations at which a highly-reflective object or other object that is likely to cause glare may be illuminated at a lower illumination intensity, e.g., to reduce the likelihood of pixel saturation in measured data associated with the object.

Dynamic sensor illumination

A sensor pod system includes one or more sensor pods with a plurality of sensors configured to collect data from an environment. A sensor pod may have an effective field of view created by individual sensors with overlapping fields of view. The sensor pod system may include sensors of different types and modalities. Sensor pods of the sensor pod system may be modularly installed on a vehicle, for example, an autonomous vehicle and collect and provide data of the environment during operation of the vehicle.

Sensor pod assembly

A lens assembly for systems such as imaging devices. The lens assembly may include at least five lens elements, where one or more of the lens elements includes positive optical power and one or more of the lenses includes negative optical power. The lens assembly may further include an aperture stop located between the lens elements. In some instances, the lens assembly provides a horizontal field of view that is at least 80 degrees while still including a total track length that is less than or equal to 50 millimeters and a diameter that is less than or equal to 13 millimeters. Additionally, in some instances, the lens assembly may cause all rays which impinge the sensor to be less than or equal to approximately 10 degrees.

Lens assembly for sensors

A LIDAR system may include a laser diode that emits a beam having a slow axis and a fast axis so that a cross-section of the beam has a width substantially greater than a height. A first three-element lens may be optically aligned with a photodetector of the LIDAR system. A second three-element lens may be optically aligned with the diode laser. The second three-element lens may include at least one lens having a predetermined astigmatism that reduces the width of the beam with respect to the height.

Lens assembly for a LIDAR system

A wind tunnel test may be performed on a vehicle to determine accumulation of substances (e.g., water) on sensors of the vehicle. Control surfaces may be created for the sensors based images representing the accumulations, where the control surfaces represent the include obstructions located where accumulations were detected during the test. The vehicle may then navigate around an environment using the control surfaces in order to determine a drivability of the vehicle. Also, a simulation may be performed, where the simulation outputs images representing simulated accumulations on the sensors. The outputs from the simulation may be compared to the results from the test in order to determine how accurately the simulation represents the test, determine domains in which the vehicle may safely operate, and/or improve the simulation.

Vehicle sensor degradation testing

Techniques for determining a degraded state associated with a sensor are discussed herein. For example, a sensor associated with vehicle may captured data of an environment. A portion of the data may represent a portion of the vehicle. Data associated with a region of interest can be determined based on a calibration associated with the sensor. For example, in the context of image data, image coordinates may be used to determine a region of interest, while in the context of lidar data, a beam and/or azimuth can be used to determine a region of interest. A data metric can be determined for data in the region of interest, and an action can be determined based on the data metric. For example, the action can include cleaning a sensor, scheduling maintenance, reducing a confidence associated with the data, or slowing or stopping the vehicle.

Sensor degradation monitor

A sensor pod system includes one or more sensor pods with a plurality of sensors configured to collect data from an environment. The sensor pod system may include a cleaning system to clean sensing surfaces of sensor pods during operation. The sensor pod system may include sensors of different types and modalities. Sensor pods of the sensor pod system may be modularly installed on a vehicle, for example, an autonomous vehicle and collect and provide data of the environment during operation of the vehicle.

Sensor pod cleaning system

A LIDAR system includes a laser emitter configured to emit a laser pulse in a sample direction of a sample area of a scene. A sensor element of the LIDAR system is configured to sense a return pulse, which is a reflection from the sample area corresponding to the emitted laser pulse. The LIDAR system may compare a width of the emitted laser pulse to a width of the return pulse in the time-domain. The comparison of the width of the emitted pulse to the width of the return pulse may be used to determine an orientation or surface normal of the sample area relative to the sample direction. Such a comparison leads to a measurement of the change of pulse width, referred to as pulse broadening or pulse stretching, from the emitted pulse to the return pulse.

Surface normal determination for LIDAR range samples by detecting probe pulse stretching

A machine-learned (ML) model for detecting that depth data (e.g., lidar data, radar data) comprises a false positive attributable to particulate matter, such as dust, steam, smoke, rain, etc. The ML model may be trained based at least in part on simulated depth data generated by a fluid dynamics model and/or by collecting depth data during operation of a device (e.g., an autonomous vehicle. In some examples, an autonomous vehicle may identify depth data that may be associated with particulate matter based at least in part on an outlier region in a thermal image. For example, the outlier region may be associated with steam.

Identifying and/or removing false positive detections from LIDAR sensor output

A sensor pod system includes one or more sensor pods with a plurality of sensors configured to collect data from an environment. A sensor pod may have an effective field of view created by individual sensors with overlapping fields of view. The sensor pod system may include sensors of different types and modalities. Sensor pods of the sensor pod system may be modularly disposed on a vehicle, for example, an autonomous vehicle to collect and provide data of the environment during operation of the vehicle. The sensor pods may be disposed at elevated locations around the vehicle to reduce obstacles within the sensor pods fields of view.

Sensor pod coverage and placement on vehicle

The described techniques relate to a simulation system that multiplexes sensor data from multiple sensors and outputs the multiplexed sensor data in channels corresponding to the multiple sensors to appropriate vehicle systems at an appropriate time based on encoded timestamp data. In examples, a multiplexer may receive sensor datasets associated with different sensors. The multiplexer may encode the different sensor datasets with timestamp data and supplemental data to generate an encoded dataset. The multiplexer may output the encoded dataset to a video output port to transmit the encoded dataset to a demultiplexer. The demultiplexer receives the encoded dataset from the video output port, and separates the encoded dataset into channels corresponding to the sensors from which the sensor data was received. The demultiplexer may output the datasets in the respective channels at a time (or times) indicated in the timestamp data and according to the supplemental data.

Multiplexing sensor data

Growing community of inventors

Mountain View, CA, United States of America

Ryan McMichael