Zoox, Inc.

Robert Nicholas Moor

Subasingha Shaminda Subasingha

Turhan Karadeniz

Joseph Patrick Warga

Ryan McMichael

Mehran Ferdowsi

Harrison Thomas Waschura

Silas Kogure Wilkinson

Yongzhe Chen

Samuel Holladay

Denis Nikitin

Shaminda Subasingha

Nathan Duarte

Jonathan Solheim

Wuyang Yu

Venkata Subrahmanyam Chandra Sekhar Chebiyyam

Jesse Sol Levinson

Haiming Wang

Till Kroeger

Daniel Glenn Johnson

George Dalman Nelson, Iii

Mohammad Umar Piracha

Maxwell Yaron

Sasidhar Saladi

Miguel Alexander Espinal

Clark Pentico

Lee Tsung

Arvind Shivu Govinday

Ryan Mcmichael

A light sensor and apparatus capable of determining depth and image data simultaneously from a target in an environment is discussed herein. The apparatus includes an array of photodetector elements comprising a first portion to detect frequency modulated continuous wave (FMCW) signals from the reflected light signals and one or more additional portions that detect visible light signals from the environment.

Lidar sensor with embedded photodetector

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can return unreliable pixels, e.g., in the case of over-exposure or saturation. In some examples, composite sensor data can be generated based on information from multiple exposures captured at different exposure times. In examples, pixels in a first exposure are matched to pixels in a second exposure to account for relative movement of the vehicle and the detected object in the composite sensor data.

Generating composite data using long and short time-of-flight exposures

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Composite sensor data can be generated based on information from multiple exposures captured at different exposure times. In examples, a first depth determination technique is applied to a first subset of pixels and a second depth determination technique is applied to a second subset of the pixels. The first subset of pixels may correspond to pixels that are associated with a highly-reflective surface.

Localized depth disambiguation for time-of-flight data

Systems and techniques for detecting feature(s) and object(s) partially or completely obscured by stray light effects in the visible light spectrum are discussed herein. Non-visible light can be emitted from emitters. Light from an environment including visible and non-visible light can be captured at a sensing system via a lens system. The sensing system can filter the captured light using a dual band-pass filter to direct certain non-visible and visible light spectra to a beam splitter that then directs the non-visible light to a non-visible light sensor and the visible light to a visible light sensor. The sensor data generated by the non-visible light sensor can then be used to identify and classify feature(s) and object(s) that may be obscured by stray light effects in the data generated by the visible light sensor.

Detecting dark objects in stray light halos

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. The sensor may generate first image data at a first configuration and second image data at a second configuration. An estimated depth of an object may be determined from the first image data, and an actual depth of the object may be determined from the second image data, based on the estimated depth. In examples, the first and second configurations have different modulation frequencies such that a nominal maximum depth in the first configuration is greater than the nominal maximum depth in the second configuration.

Determining depth using multiple modulation frequencies

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can be impacted by glare. In some examples, an emitter system for illuminating a field of view can be dynamically altered to provide different illumination intensities at different regions in the field of view. For instance, locations at which a highly-reflective object or other object that is likely to cause glare may be illuminated at a lower illumination intensity, e.g., to reduce the likelihood of pixel saturation in measured data associated with the object.

Dynamic sensor illumination

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can include returns associated with highly reflective objects that cause glare. In some examples, a depth of a sensed surface is determined from the sensor data and additional pixels at the same depth are identified. The subset of pixels at the depth are filtered by comparing a measured intensity value to a threshold intensity value for the depth. Other threshold intensity values can be applied to subsets of pixels at different depths.

Depth dependent pixel filtering

A lens assembly for systems such as imaging devices. The lens assembly may include at least five lens elements, where one or more of the lens elements includes positive optical power and one or more of the lenses includes negative optical power. The lens assembly may further include an aperture stop located between the lens elements. In some instances, the lens assembly provides a horizontal field of view that is at least 80 degrees while still including a total track length that is less than or equal to 50 millimeters and a diameter that is less than or equal to 13 millimeters. Additionally, in some instances, the lens assembly may cause all rays which impinge the sensor to be less than or equal to approximately 10 degrees.

Lens assembly for sensors

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. The sensor may generate first image data at a first configuration and second image data at a second configuration. A disambiguated depth of a surface may be determined from the first image data and the second image data. If the disambiguated depth is greater than a nominal maximum depth of the sensor in the first configuration, an intensity of the surface may be determined from the first image data. If the intensity meets or exceeds a threshold intensity, the surface may be determined to be beyond the nominal maximum depth. If the intensity is less than the threshold intensity, an actual depth of the surface may be determined form the second image data as a distance less than the nominal maximum depth.

Determining pixels beyond nominal maximum sensor depth

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can include saturated pixels, e.g., due to over-exposure, sensing highly-reflective objects, and/or excessive ambient light. In some examples, parameters associated with power of a time-of-flight sensor can be altered based on characteristics of the saturated pixels, as well as information about non-saturated pixels neighboring the saturated pixels. For example, the neighboring pixels may provide information about whether saturation is due to ambient light, e.g., sunlight, or due to emitted light from the sensor.

Active power control using neighboring pixels

A vehicle mounted sensor assembly includes a frame and a cover connected to the base. When assembled, the frame and the cover define a volume that is divided between a first cavity and a second cavity. A sensor is disposed in the first cavity and one or more antennas are disposed in the second cavity.

Vehicle-mounted sensor and antenna assembly

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can return unreliable pixels, e.g., in the case of over- or under-exposure. In some examples, parameters associated with power of a time-of-flight sensor can be altered based on a number of unreliable pixels in measured data and/or based on intensity values of the measured data. For example, unreliable pixels can be determined using phase frame information captured at a receiver of the sensor.

Active power control of sensors

Various lens designs for use in Time of Flight (ToF) sensor systems are discussed. Improvements to a ToF sensor system may be realized by, for example, incorporating a lens having particular features, such as a relatively short track length, fast lens speed (e.g., low f-number), low telecentricity, relatively flat field illumination, and fairly low cost. In some examples, such a lens of a ToF sensor system may be a lens assembly having a fixed focal length and that avoids use of lenses having aspheric surfaces so as to achieve relatively low cost.

Time of flight sensor system and lens

A blemish detection and characterization system and techniques for an optical imaging device includes determining a ratio of the light intensity of the image lost to the blemish relative to an expected light intensity of the image without the blemish. The system and technique may include receiving an image, transforming an image into a processed image with transformations and filters, as well as determining a relative magnitude of an intensity of a portion of the processed image relative to another area of the image. The system and technique may include taking an action based on the relative magnitude including rejecting a sensor, reworking the sensor, cleaning the sensor, or providing information about the blemish to a system to use in weighing data collected from the sensor.

Imaging device blemish detection structures and techniques

An imaging device blemish detection test enclosure and techniques for an optical imaging device includes a mounting structure for mounting an optical imaging device, a first body with a concave surface, and a second body holding the mounting structure relative to the first body. The mounting structure and the second body may orient an optical axis of a lens of the optical imaging device towards the concave surface and locate the lens relative to the concave surface where the interface between the first and second bodies is outside of a lens field of view of the lens. The system may include a light source disposed in the second body and directed towards the concave surface of the of the first body providing an evenly illuminating the concave surface. The concave surface may include a surface of a spherical sector greater than a hemisphere.

Growing community of inventors

San Mateo, CA, United States of America

Robert Nicholas Moor