University of Michigan

Toyota Research Institute, Inc.

Ford Global Technolgoies, LLC

Ryan M Eustice

Edwin B Olson

Ryan W Wolcott

Michael R James

Jeffrey Michael Walls

James McBride

Steven A Parkison

Enric Galceran

Alexander Glencairn Cunningham

Mohammad Saad

A system for determining the rigid-body transformation between 2D image data and 3D point cloud data includes a first sensor configured to capture image data of an environment, a second sensor configured to capture point cloud data of the environment; and a computing device communicatively coupled to the first sensor and the second sensor. The computing device is configured to receive image data from the first sensor and point cloud data from the second sensor, parameterize one or more 2D lines from image data, parameterize one or more 3D lines from point cloud data, align the one or more 2D lines with the one or more 3D lines by solving a registration problem formulated as a mixed integer linear program to simultaneously solve for a projection transform vector and a data association set, and generate a data mesh comprising the image data aligned with the point cloud data.

2D to 3D line-based registration with unknown associations

A system and method for calibrating sensors may include one or more processors, a first sensor configured to obtain a two-dimensional image, a second sensor configured to obtain three-dimensional point cloud data, and a memory device. The memory device stores a data collection module and a calibration module. The data collection module has instructions that configure the one or more processors to obtain the two-dimensional image and the three-dimensional point cloud data. The calibration module has instructions that configure the one or more processors to determine and project a three-dimensional point cloud edge of the three-dimensional point cloud data onto the two-dimensional image edge, apply a branch-and-bound optimization algorithm to a plurality of rigid body transforms, determine a lowest cost transform of the plurality of rigid body transforms using the branch-and-bound optimization algorithm, and calibrate the first sensor with the second sensor using the lowest cost transform.

System and method for calibrating sensors of a sensor system

System, methods, and other embodiments described herein relate to identifying changes between models of a locality. In one embodiment, a method includes, in response to determining that a location model is available for a present environment of a vehicle, generating a current model of the present environment using at least one sensor of the vehicle. The method also includes isolating dynamic objects in the current model as a function of the location model. The method includes providing the dynamic objects to be identified and labeled.

Systems and methods for identifying changes within a mapped environment

System, methods, and other embodiments described herein relate to identifying surface properties of objects using a light detection and ranging (LIDAR) sensor. In one embodiment, a method includes, in response to scanning a surface of an object using the LIDAR sensor, receiving a reflected waveform as a function of attributes of the surface. The method includes analyzing the reflected waveform according to a surface property model to produce an estimate of the attributes. The surface property model characterizes relationships between reflected waveforms and different surface properties. The method includes providing the estimate as an indication of the surface of the scanned object.

Systems and methods for surface property identification using waveform classification

An apparatus and method for visual localization of a visual camera system outputting real-time visual camera data and a graphics processing unit receiving the real-time visual camera data. The graphics processing unit accesses a database of prior map information and generates a synthetic image that is then compared to the real-time visual camera data to determine corrected position data. The graphics processing unit determines a camera position based on the corrected position data. A corrective system for applying navigation of the vehicle based on the determined camera position can be used in some embodiments.

Visual localization within LIDAR maps

A system includes a computer programmed to identify, from a first vehicle, one or more second vehicles within a specified distance to the first vehicle. The computer is further programmed to receive data about operations of each of the second vehicles, including trajectory data. Based on the data, the computer is programmed to identify, for each of the second vehicles, a distribution of probabilities of each of a set of potential planned trajectories. The computer is further programmed to determine a planned trajectory for the first vehicle, based on the respective distributions of probabilities of each of the set of potential planned trajectories for each of the second vehicles. The computer is further programmed to provide an instruction to at least one controller associated with the first vehicle based on the determined planned trajectory.

Vehicle trajectory determination

A system includes a computer programmed to determine, along a nominal path to be traversed by a vehicle, a potential field representing a driving corridor for the vehicle. The computer is further programmed to identify a position of the vehicle relative to the potential field at a current time, and apply a torque to q steering column of the vehicle. The torque is based at least in part on the position. The potential field includes an attractive potential that guides the vehicle to remain within the corridor.

Growing community of inventors

Ann Arbor, MI, United States of America

Ryan M Eustice