Zoox, Inc.

Joshua Kriser Cohen

Chuang Wang

Subasingha Shaminda Subasingha

Cooper Stokes Sloan

Venkata Subrahmanyam Chandra Sekhar Chebiyyam

Samantha Marie Ting

Badeea Ferdaous Alferdaous Alazem

Sabeek Mani Pradhan

Jacob Daniel Boydston

Yu Liu

Anton Mario Bongio Karrman

Balazs Kovacs

James William Vaisey Philbin

Michael Carsten Bosse

Jifei Qian

Yassen Ivanchev Dobrev

Yuesong Xie

Dilip Bethanabhotla

Yuan Zhang

Hang Ren

Techniques are discussed herein for analyzing radar data to determine that radar noise from one or more target detections potentially conceals additional objects near the target detection. Determining whether an object may be concealed can be based at least in part on a radar noise level based on a target detection, as well as distributions of radar cross sections and/or doppler data associated with particular object types. For a location near a target detection, a radar system may determine estimated noise levels, and compare the estimated noise levels to radar cross section probabilities associated with object types to determine the likelihood that an object of the object type could be concealed at the location. Based on the analysis, the system may determine a vehicle trajectory or otherwise may control a vehicle based on the likelihood that an object may be concealed at the location.

Radar data analysis and concealed object detection

Techniques are discussed for determining reflected returns in radar sensor data. In some instances, pairs of radar returns may be compared to one another. For example, a reflection point may be determined from a first position of a first radar return and a second position of a second radar return. Additional data, e.g., sensor data and/or map data, may be used to determine the presence of objects in the environment. The first return or the second return may be a reflected return if an object is disposed at the reflection point. In some instances, a vehicle, such as an autonomous vehicle, may be controlled at the exclusion of information from reflected returns.

Recognizing radar reflections using position information

Techniques for determining a probability of a false negative associated with a location of an environment are discussed herein. Data from a sensor, such as a radar sensor, can be received that includes point cloud data, which includes first and second data points. The first data point has a first attribute and the second data point has a second attribute. A difference between the first and second attributes is determined such that a frequency distribution may be determined. The frequency distribution may then be used to determine a distribution function, which allows for the determination of a resolution function that is associated with the sensor. The resolution function may then be used to determine a probability of a false negative at a location in an environment. The probability can be used to control a vehicle in a safe and reliable manner.

Data driven resolution function derivation

A machine-learning architecture may be trained to determine point cloud data associated with different types of sensors with an object detected in an image and/or generate a three-dimensional region of interest (ROI) associated with the object. In some examples, the point cloud data may be associated with sensors such as, for example, a lidar device, radar device, etc.

Multi-modal sensor data association architecture

Techniques for updating data operations in a perception system are discussed herein. A vehicle may use a perception system to capture data about an environment proximate to the vehicle. The perception system may receive state data stored in cyclic buffer of globally registered detection and occasionally converted to gridded point cloud in a local reference frame. The two-dimensional gridded point cloud may be processed using one or more neural networks to generate semantic data associated with a scene or physical environment surrounding the vehicle such that the vehicle can make environment aware operational decisions, which may improve reaction time(s) and/or safety outcomes of the autonomous vehicle.

Perception system

A collision avoidance system may validate, reject, or replace a trajectory generated to control a vehicle. The collision avoidance system may comprise a secondary perception component that may receive sensor data, receive and/or determine a corridor associated with operation of a vehicle, classify a portion of the sensor data associated with the corridor as either ground or an object, determine a position and/or velocity of at least the nearest object, determine a threshold distance associated with the vehicle, and control the vehicle based at least in part on the position and/or velocity of the nearest object and the threshold distance.

Collision avoidance perception system

A tool for providing vehicle perception system metrics associated with objects that are relevant to the vehicle is discussed herein. The tool may be configured with criteria for relevance. The criteria may include a region of interest proximate the vehicle, an object classification, an object characteristic and/or a sensor type. The tool may receive sensor data captured by a sensor of the vehicle and may be configured to identify relevant objects based on the criteria. The tool may filter and/or determine perception system metrics associated with the relevant objects. In some examples, the tool may identify the relevant objects and may cause the vehicle to be controlled based on the determination of object relevance.

Relevant object detection

Classifying sensor data as being associated with ground (as opposed to an object) may comprise determining a number of channels of sensor data that have returns in them, setting a number of control points and a number of knots of a curve based at least in part on the number of channels that have returns, and fitting a curve having the number of control points and the number of knots to the sensor data. The curve may be used to distinguish sensor data associated with the ground from sensor data associated with an object. Determining the curve may additionally or alternatively include limiting an elevation value of a control point and/or knot based on elevation value(s) of the sensor data, weighting the sensor data based at least in part on elevation values associated with the sensor data, and/or adjusting knot spacing, et alia.

Complex ground profile estimation

Techniques are discussed for generating a spatial grid based on radar sensor data. The spatial grid may include cells, which may be designated as being occupied, occluded, and/or free space. A cell may be designated as being occupied if radar sensor data is associated with a region of an environment associated with the cell. A field may be designated as being occluded if a region of the environment associated with the cell is obstructed, relative to a radar sensor, by a cell that is designated as being occupied. In some instances, objects may be detected and/or tracked in occluded regions. A cell may be designated as being free space if a region associated with the cell is within a field of view of a radar sensor and is unoccupied and un-occluded. In some instances, the spatial grid may be used to control a vehicle, such as an autonomous vehicle, in the environment.

Radar spatial estimation

Techniques are discussed for determining reflected returns in radar sensor data. In some instances, pairs of radar returns may be compared to one another. For example, a velocity associated with a first radar return may be projected onto a radial direction associated with a second radar return to determine a projected velocity. In some examples, the second radar return may be a reflected return if the magnitude of the projected velocity corresponds to a magnitude of the second radar return. In some instances, a vehicle, such as an autonomous vehicle, may be controlled at the exclusion of information from reflected returns.

Recognizing radar reflections using velocity information

Techniques are described for determining a likelihood that a radar device failed to detect an object (i.e., a false negative). Determining the likelihood may be based at least in part on determining an estimated noise floor based at least in part on at least a portion of radar data, which may comprise one or more detections, and determining a likelihood that the portion of radar data includes a false positive, based at least in part on the estimated noise floor and a response profile associated with an object. A response profile may identify a received signal power and/or radar cross section associated with an object type.

Radar false negative analysis

Sensors, including radar sensors, may be used to detect objects in an environment. In an example, a vehicle may include multiple radar sensors that sense objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Data from a first of the radar sensors can be analyzed to determine a cluster representing an object. Data received from a second sensor (and additional sensors, as included) at substantially the same time can be analyzed to determine additional points to include in the cluster. In additional examples, the radar sensors can have a different interval, e.g., Doppler interval, and information about the points and the intervals can be used to determine a speed of the object relative to the vehicle.

Growing community of inventors

Sunnyvale, CA, United States of America

Joshua Kriser Cohen