Denso Corporation

Denso International America, Inc.

Carnegie Mellon University

Prasanna Sivakumar

Shawn T Hunt

Kris Kitani

Matthew O'Toole

Xinshuo Weng

Minglei Huang

Yu-Jhe Li

Matthew Patrick O'toole

Yunze Man

Systems, methods, and other embodiments described herein relate to enhancing an output of a LiDAR sensor. In one embodiment, a method includes acquiring, using the LiDAR sensor, a point cloud and an ambient image. The method includes processing the point cloud and the ambient image to generate enhanced data according to an enhancer pipeline that includes a set of processing routines. The method includes providing the enhanced data to at least one module to facilitate generating determinations about a surrounding environment of the LiDAR sensor from the enhanced data.

Systems and methods for enhancing outputs of a LiDAR

A method includes generating a radar-based intensity map and a lidar-based intensity map and performing one or more augmentation routines on the radar-based intensity map and the lidar-based intensity map to generate a radar input and a lidar input. The method includes generating a plurality of teacher-based bounding boxes and a plurality of student-based bounding boxes based on the radar input and the lidar input. The method includes determining a loss value of the plurality of student-based bounding boxes based on the plurality of teacher-based bounding boxes and a plurality of ground truth bounding boxes, updating one or more weights of the student neural network based on the loss value, and updating one or more weights of the teacher neural network based on a moving average associated with the one or more weights of the student neural network.

System and method for training a neural network to perform object detection using lidar sensors and radar sensors

A method includes generating a plurality of lidar inputs based on the lidar data, where each lidar input from among the plurality of lidar inputs comprises an image-based portion and a geometric-based portion, and where each lidar input from among the plurality of lidar inputs defines a position coordinate of the one or more objects. The method includes performing, for each lidar input from among the plurality of lidar inputs, a convolutional neural network (CNN) routine based on the image-based portion to generate one or more image-based outputs and assigning the plurality of lidar inputs to a plurality of echo groups based on the geometric-based portion. The method includes concatenating the one or more image-based outputs and the plurality of echo groups to generate a plurality of fused outputs and identifying the one or more objects based on the plurality of fused outputs.

Systems and methods for detecting objects based on lidar data

A method of calibrating a camera sensor and a SPAD LiDAR includes extracting identified features in each of a selected camera image and an ambient-intensity (A-I) image, generating a set of keypoints based on the identified features extracted for each of the images to provide a set of 2D camera keypoint locations and a set of 2D A-I keypoint locations, determining matched keypoints based on the set of 2D A-I keypoint locations and the set of 2D camera keypoint locations to provide a set of 2D A-I matched pixel locations and a set of 2D camera matched pixel locations, interpolating a 3D point cloud data with the set of 2D A-I matched pixel locations to obtain a set of 3D LiDAR matched pixel locations, and determining and storing extrinsic parameters to transform the set of 3D LiDAR matched pixel locations with the set of 2D camera matched pixel locations.

Method and system for automatic extrinsic calibration of SPAD LiDAR and camera pairs

A method of generating virtual sensor data of a virtual single-photon avalanche diode (SPAD) lidar sensor includes generating a two-dimensional (2D) lidar array having a plurality of cells. The method further includes interpolating image data from a virtual camera with the 2D lidar array to define auxiliary image data, generating a virtual ambient image based on a red-channel (R-channel) data of the auxiliary image data, identifying a plurality of virtual echoes of the virtual SPAD lidar sensor based on the R-channel data and a defined photon threshold, defining a virtual point cloud indicative of virtual photon measurements of the virtual SPAD lidar sensor based on the plurality of virtual echoes, and outputting data indicative of the virtual ambient image, the virtual photon measurements, the virtual point cloud, or a combination thereof, as the virtual sensor data of the virtual SPAD lidar sensor.

Methods and system for generating virtual sensor data of a virtual single-photon avalanche diode (SPAD) lidar sensor of a virtual vehicle simulator

A method includes obtaining steering data of the vehicle from one or more steering sensors, determining whether the vehicle is operating in one of a turning state and a non-turning state based on the steering data, performing a localization routine based on a first echo set of the 3D data points from among the plurality of 3D data points when the vehicle is operating in the turning state, and performing the localization routine based on a second echo set of the 3D data points from among the plurality of 3D data points when the vehicle is operating in the non-turning state, where a number of the plurality of 3D data points of the first echo set is greater than a number of the plurality of 3D data points of the second echo set.

Systems and methods for controlling a vehicle localization routine

A target-orientated navigation system and related method for a vehicle having a generic navigation system includes one or more processors and a memory. The memory includes one or more modules that cause the processor to receive perception data, discretize the perception data into a plurality of lattices, generate a collision probability array having a plurality of cells that correspond to the plurality of lattices, determine which cells of the collision probability array satisfy a safety criteria, receive an artificial potential field array having a plurality of cells that correspond to the plurality of cells of the collision probability array, generate, an objective score array having a plurality of cells corresponding to the cells of the collision probability array, and direct a vehicle control system of the vehicle to guide the vehicle to a location representative of a cell in the objective score array that has a highest value.

Target-orientated navigation system for a vehicle using a generic navigation system and related method

An artificial intelligence system for an autonomous vehicle includes one or more processors and a memory in communication with the one or more processors and storing a target-orientated navigation system module. When executed, the target-orientated navigation system module causes the one or more processors to receive sensor data from one or more sensors of a vehicle, synchronize the sensor data, preprocess the synchronized sensor data by transforming the sensor data into a common data format, concatenate the transformed sensor data into a K-dimensional array, which acts as an input state array, apply a navigation policy to the input state array to estimate an action-value array, direct a vehicle control system to guide the vehicle to a location representative of a cell in the action-value array that has a highest reward value.

Growing community of inventors

Pittsburgh, PA, United States of America

Prasanna Sivakumar