Ford Global Technolgoies, LLC

Michigan State University

Marcos Paul Gerardo Castro

Jinesh J Jain

Sneha Kadetotad

Dongran Liu

Vidya Nariyambut Murali

Adil Nizam Siddiqui

Punarjay Chakravarty

Bruno Sielly Jales Costa

Praveen Narayanan

Xiaoming Liu

Daniel Morris

Yunfei Long

Ashley Elizabeth Micks

Shreyasha Paudel

Jinhyoung Oh

Clifton Thomas

Feng Jin

Sandhya Bhaskar

Tracking an object included in a reflective surface by detecting the reflective surface included in one or more images included in a plurality of images by determining a location of the reflective surface in pixel coordinates and tracking the location of the reflective surface in the plurality of images. Real world locations of the object can be determined based on attributes of the reflective surface including a geometric class, extrinsic properties that relate the reflective surface to an environment, intrinsic properties describing the reflective surface without referring to the environment and calibration properties of the sensor.

Object detection using reflective surfaces

A computer includes a processor and a memory storing instructions executable by the processor to receive radar data including a radar pixel having a radial velocity from a radar; receive camera data including an image frame including camera pixels from a camera; map the radar pixel to the image frame; generate a region of the image frame surrounding the radar pixel; determine association scores for the respective camera pixels in the region; select a first camera pixel of the camera pixels from the region, the first camera pixel having a greatest association score of the association scores; and calculate a full velocity of the radar pixel using the radial velocity of the radar pixel and a first optical flow at the first camera pixel. The association scores indicate a likelihood that the respective camera pixels correspond to a same point in an environment as the radar pixel.

Vehicle using full-velocity determination with radar

A computer, including a processor and a memory, the memory including instructions to be executed by the processor to, based on sensor data in a vehicle, determine a database that includes object data for a plurality of objects, including, for each object, an object identification, a measurement of one or more object attributes, and an uncertainty specifying a probability of correct object identification, for the object identification and the object attributes determined based on the sensor data, wherein the object attributes include an object size, an object shape and an object location. The instructions include further instructions to determine a map based on the database including the respective locations and corresponding uncertainties for the vehicle type and download the map to a vehicle based on the vehicle location and the vehicle type.

Vehicle uncertainty sharing

A computer includes a processor and a memory storing instructions executable by the processor to receive radar data from a radar, the radar data including radar pixels having respective measured depths; receive camera data from a camera, the camera data including an image frame including camera pixels; map the radar pixels to the image frame; generate respective regions of the image frame surrounding the respective radar pixels; for each region, determine confidence scores for the respective camera pixels in that region; output a depth map of projected depths for the respective camera pixels based on the confidence scores; and operate a vehicle including the radar and the camera based on the depth map. The confidence scores indicate confidence in applying the measured depth of the radar pixel for that region to the respective camera pixels.

Depth map generation

Systems, methods, and devices for estimating a shape of an object based on sensor data and determining a presence of a fault or a failure in a perception system. A method of the disclosure includes receiving sensor data from a range sensor and calculating a current shape reconstruction of an object based on the sensor data. The method includes retrieving from memory a prior shape reconstruction of the object based on prior sensor data. The method includes calculating a quality score for the current shape reconstruction by balancing a function of resulted variances of the current shape reconstruction and a similarity between the current shape reconstruction and the prior shape reconstruction.

Simultaneous diagnosis and shape estimation from a perceptual system derived from range sensors

A system can include a computer including a processor and a memory, the memory storing instructions executable by the processor to receive point cloud data. The instructions further include instructions to generate a plurality of feature maps based on the point cloud data, each feature map of the plurality of feature maps corresponding to a parameter of the point cloud data. The instructions further include instructions to aggregate the plurality of feature maps into an aggregated feature map. The instructions further include instructions to generate, via a feedforward neural network, at least one of a segmentation output or a classification output based on the aggregated feature map.

Segmentation and classification of point cloud data

A computer includes a processor and a memory storing instructions executable by the processor to collect a plurality of data sets, each data set from a respective sensor in a plurality of sensors, and each data set including a range, an azimuth angle, and a range rate for a detection point of the respective one of the sensors on an object to determine, for each detection point, a radial component of a ground speed of the detection point based on the data set associated with the detection point and a speed of a vehicle, and to generate a plurality of clusters, each cluster including selected detection points within a distance threshold from each other and having respective radial components of ground speeds that are (1) above a first threshold and (2) within a second threshold of each other.

Enhanced object detection with clustering

A controller receives outputs from a plurality of sensors such as a camera, LIDAR sensor, RADAR sensor, and ultrasound sensor, which may be rearward facing. A probability is updated each time a feature in a sensor output indicates presence of an object. The probability may be updated as a function of a variance of the sensor providing the output and a distance to the feature. Where the variance of a sensor is directional, directional probabilities may be updated according to these variances and the distance to the feature. If the probability meets a threshold condition, actions may be taken such as a perceptible alert or automatic braking. The probability may be decayed in the absence of detection of objects. Increasing or decreasing trends in the probability may be amplified by further increasing or decreasing the probability.

Multi-sensor probabilistic object detection and automated braking

A system for detecting and identifying foliage includes a tracking component, a tracking parameters component, and a classification component. The tracking component is configured to detect and track one or more features within range data from one or more sensors. The tracking parameters component is configured to determine tracking parameters for each of the one or more features. The tracking parameters include a tracking age and one or more of a detection consistency and a position variability. The classification component is configured to classify a feature of the one or more features as corresponding to foliage based on the tracking parameters.

Detecting foliage using range data

A scenario is defined that including models of vehicles and a typical driving environment. A model of a subject vehicle is added to the scenario and sensor locations are defined on the subject vehicle. Perception of the scenario by sensors at the sensor locations is simulated to obtain simulated sensor outputs. The simulated sensor outputs are annotated to indicate the location of obstacles in the scenario. The annotated sensor outputs may then be used to validate a statistical model or to train a machine learning model. The simulates sensor outputs may be modeled with sufficient detail to include sensor noise or may include artificially added noise to simulate real world conditions.

Generating simulated sensor data for training and validation of detection models

Example obstacle detection systems and methods are described. In one implementation, a method receives data from at least one sensor mounted to a vehicle and creates a probabilistic grid-based map associated with an area near the vehicle. The method also determines a confidence associated with each probability in the grid-based map and determines a likelihood that an obstacle exists in the area near the vehicle based on the probabilistic grid-based map.

Obstacle detection systems and methods

A controller receives outputs form a plurality of sensors such as a camera, LIDAR sensor, RADAR sensor, and ultrasound sensor. Sensor outputs corresponding to an object are assigned to a tracklet. Subsequent outputs by any of the sensors corresponding to that object are also assigned to the tracklet. A trajectory of the object is calculated from the sensor outputs assigned to the tracklet, such as by means of Kalman filtering. For each sensor output assigned to the tracklet, a probability is updated, such as using a Bayesian probability update. When the probability meets a threshold condition, the object is determined to be present and an alert is generated or autonomous obstacle avoidance is performed with respect to an expected location of the object.

Growing community of inventors

Mountain View, CA, United States of America

Marcos Paul Gerardo Castro