Carnegie Mellon University

Denso Corporation

University of Toronto

Matthew O'Toole

Srinivasa Narasimhan

Shawn T Hunt

Kiriakos Neoklis Kutulakos

Prasanna Sivakumar

Supreeth Achar

Mark Sheinin

Kris Kitani

Xinshuo Weng

Yunze Man

Yu-Jhe Li

A novel class of imaging systems that combines diffractive optics with 1D line sensing is disclosed. When light passes through a diffraction grating or prism, it disperses as a function of wavelength. This property is exploited to recover 2D and 3D positions from line images. A detailed image formation model and a learning-based algorithm for 2D position estimation are disclosed. The disclosure includes several extensions of the imaging system to improve the accuracy of the 2D position estimates and to expand the effective field-of-view. The invention is useful for fast passive imaging of sparse light sources, such as streetlamps, headlights at night and LED-based motion capture, and structured light 3D scanning with line illumination and line sensing.

Systems and methods for diffraction line imaging

A method includes generating a radar-based intensity map and a lidar-based intensity map and performing one or more augmentation routines on the radar-based intensity map and the lidar-based intensity map to generate a radar input and a lidar input. The method includes generating a plurality of teacher-based bounding boxes and a plurality of student-based bounding boxes based on the radar input and the lidar input. The method includes determining a loss value of the plurality of student-based bounding boxes based on the plurality of teacher-based bounding boxes and a plurality of ground truth bounding boxes, updating one or more weights of the student neural network based on the loss value, and updating one or more weights of the teacher neural network based on a moving average associated with the one or more weights of the student neural network.

System and method for training a neural network to perform object detection using lidar sensors and radar sensors

A method of generating virtual sensor data of a virtual single-photon avalanche diode (SPAD) lidar sensor includes generating a two-dimensional (2D) lidar array having a plurality of cells. The method further includes interpolating image data from a virtual camera with the 2D lidar array to define auxiliary image data, generating a virtual ambient image based on a red-channel (R-channel) data of the auxiliary image data, identifying a plurality of virtual echoes of the virtual SPAD lidar sensor based on the R-channel data and a defined photon threshold, defining a virtual point cloud indicative of virtual photon measurements of the virtual SPAD lidar sensor based on the plurality of virtual echoes, and outputting data indicative of the virtual ambient image, the virtual photon measurements, the virtual point cloud, or a combination thereof, as the virtual sensor data of the virtual SPAD lidar sensor.

Methods and system for generating virtual sensor data of a virtual single-photon avalanche diode (SPAD) lidar sensor of a virtual vehicle simulator

An energy optimized imaging system that includes a light source that has the ability to illuminate specific pixels in a scene, and a sensor that has the ability to capture light with specific pixels of its sensor matrix, temporally synchronized such that the sensor captures light only when the light source is illuminating pixels in the scene.

Energy optimized imaging system with synchronized dynamic control of directable beam light source and reconfigurably masked photo-sensor

Imaging system with synchronized dynamic control of directable beam light source and reconfigurably masked photo-sensor

Growing community of inventors

Palo Alto, CA, United States of America

Matthew O'Toole