Aptiv Technologies Limited

Delphi Technologies, Inc.

Aptiv Technologies Ag

Izzat H Izzat

Yang Zheng

Dimitris Zermas

Anuradha Mangalgiri

Ronald Melvin Taylor

Ping Yuan

Susan Chen

Uday Pitambare

Sanling Song

Rohith Mv

Uday S Pitambare

This document describes methods and systems directed at imaging sensor and radar fusion for multiple-object tracking. Using tracking-by-detection, an object is first detected in a frame captured by an imaging sensor, and then the object is tracked over several consecutive frames by both the imaging sensor and a radar system. The object is tracked by assigning a probability that the object identified in one frame is a same object identified in the consecutive frame. A probability is calculated for each data set captured by a sensor by a supervised-learning neural-network model using the data collected from the sensors. Then, the probabilities associated with each sensor are fused into a refined probability. By fusing the data gathered by the imaging sensor and the radar system in the consecutive frames, a safety system can track multiple objects more accurately and reliably than using the sensor data separately to track objects.

Imaging and radar fusion for multiple-object tracking

This document describes 'Multi-domain Neighborhood Embedding and Weighting' (MNEW) for use in processing point cloud data, including sparsely populated data obtained from a lidar, a camera, a radar, or combination thereof. MNEW is a process based on a dilation architecture that captures pointwise and global features of the point cloud data involving multi-scale local semantics adopted from a hierarchical encoder-decoder structure. Neighborhood information is embedded in both static geometric and dynamic feature domains. A geometric distance, feature similarity, and local sparsity can be computed and transformed into adaptive weighting factors that are reapplied to the point cloud data. This enables an automotive system to obtain outstanding performance with sparse and dense point cloud data. Processing point cloud data via the MNEW techniques promotes greater adoption of sensor-based autonomous driving and perception-based systems.

Multi-domain neighborhood embedding and weighting of sampled data

Multi-domain neighborhood embedding and weighting of point cloud data

An object detection system includes color and infrared cameras, a controller-circuit, and instructions. The color and infrared cameras are configured to output respective color image and infrared image signals. The controller-circuit is in communication with the cameras, and includes a processor and a storage medium. The processor is configured to receive and transform the color image and infrared image signals into classification and location data associated with a detected object. The instructions are stored in the at least one storage medium and executed by the at least one processor, and are configured to utilize the color image and infrared image signals to form respective first and second maps. The first map has a first plurality of layers, and the second map has a second plurality of layers. Selected layers from each are paired and fused to form a feature pyramid that facilitates formulation of the classification and location data.

Object detection system of a vehicle

An object detection system includes a lidar-unit and a controller. The controller defines an occupancy-grid that segregates the field-of-view into columns, determine a first-occupancy-status of a column based on first-cloud-points detected by the lidar-unit in the column by a first-scan, determine a second-occupancy-status of the column based second-cloud-points detected in the column by a second-scan, determine a first-number of the first-cloud-points and a second-number of the second-cloud-points, and determine a dynamic-status of the column only if the column is classified as occupied by either the first-occupancy-status or the second-occupancy-status. The dynamic-status of the column is determined to be moving when a count-difference between the first-number and the second-number is greater than a difference-threshold, and the dynamic-status of the column is determined to be static when the count-difference is not greater than the difference-threshold and a registration-factor that aligns the first-cloud-points to the second-cloud-points is less than a registration-threshold.

Object detection system

A ground-classifier system that classifies ground-cover proximate to an automated vehicle includes a lidar, a camera, and a controller. The lidar that detects a point-cloud of a field-of-view. The camera that renders an image of the field-of-view. The controller is configured to define a lidar-grid that segregates the point-cloud into an array of patches, and define a camera-grid that segregates the image into an array of cells. The point-cloud and the image are aligned such that a patch is aligned with a cell. A patch is determined to be ground when the height is less than a height-threshold. The controller is configured to determine a lidar-characteristic of cloud-points within the patch, determine a camera-characteristic of pixels within the cell, and determine a classification of the patch when the patch is determined to be ground, wherein the classification of the patch is determined based on the lidar-characteristic and the camera-characteristic.

Ground classifier system for automated vehicles

An illustrative example object detection system includes a camera having a field of view. The camera provides an output comprising information regarding potential objects within the field of view. A processor is configured to select a portion of the camera output based on information from at least one other type of detector that indicates a potential object in the selected portion. The processor determines an Objectness of the selected portion based on information in the camera output regarding the selected portion.

Enhanced camera object detection for automated vehicles

An object-detection system includes a radar sensor, a camera, and a controller. The radar-sensor is suitable for mounting on a vehicle and is used to detect a radar-signal reflected by an object in a radar-field-of-view. The radar-signal is indicative of a range, range-rate, and a direction to the object relative to the vehicle. The camera is used to capture an image of a camera-field-of-view that overlaps the radar-field-of-view. The controller is in communication with the radar-sensor and the camera. The controller is configured to determine a range-map for the image based on the range and the direction of the radar detection, define a detection-zone in the image based on the range-map, and process only the detection-zone of the image to determine an identity of the object.

Object detection using radar and vision defined image detection zone

A road-model-definition system suitable for an automated-vehicle includes a lidar-unit and a controller. The lidar-unit is suitable to mount on a host-vehicle. The lidar-unit is used to provide a point-cloud descriptive of an area proximate to the host-vehicle. The controller is in communication with the lidar-unit. The controller is configured to: select ground-points from the point-cloud indicative of a travel-surface, tessellate a portion of the area that corresponds to the travel-surface to define a plurality of cells, determine an orientation of each cell based on the ground-points within each cell, define a road-model of the travel-surface based on the orientation of the cells, and operate the host-vehicle in accordance with the road-model.

Automated vehicle road model definition system

An object-detection system suitable for an automated vehicle includes a lidar and a controller. The lidar is used to detect a point-cloud that is organized into a plurality of scan-lines. The controller is in communication with the lidar. The controller is configured to classify each detected point in the point-cloud as a ground-point or a non-ground-point, define runs of non-ground-points, where each run characterized by one or multiple instances of adjacent non-ground-points in a scan-line separated from a subsequent run of one or more non-ground-points by at least one instance of a ground-point, define a cluster of non-ground-points associated with the object. The cluster is characterized by a first run from a first scan-line being associated with a second run from a second scan-line when a first point from the first run is displaced less than a distance-threshold from a second point from the second run.

Growing community of inventors

Oak Park, CA, United States of America

Izzat H Izzat