Luminar Technologies, Inc.

Luminar, LLC

Richmond Hicks

Jason M Eichenholz

Matthew D Weed

A machine vision system comprises a camera configured to generate one or more images of a field of regard of the camera, a lidar system, and a processor. The lidar system includes a laser configured to emit light, where the emitted light is directed toward a region within the field of regard of the camera and a receiver configured to detect light returned from the emitted light. The processor is configured to receive an indication of a location based on the returned light and determine, based on the one or more images generated by the camera, whether the indication of the location is associated with a spurious return.

Camera-gated lidar system

The bit depth of the pixels in a camera image are reduced. In one embodiment, the number of pixels for each of a set of multiple bit intensity values is counted. A pixel intensity value with a lowest count is selected and a mapping function is generated that combines the pixels with the lowest count with pixels having an adjacent pixel intensity value. This is repeated until a total number of pixel intensity value counts is reduced to a predetermined number. A reduced bit depth image is generated using the predetermined number of pixel intensity value counts by assigning a new pixel intensity value to each of the pixels using the mapping function and the reduced bit depth image is sent to an image analysis system.

Bit depth reduction of image pixels

Lidar return data is fused early with camera information. In one embodiment, a voxel is detected in a frame of a lidar corresponding to an external scene. The voxel is mapped to a patch in a frame of a camera of the external scene corresponding to the voxel. The patch has a plurality of pixels. Each pixel has a plurality of color values. The voxel is augmented with the patch of pixels and the augmented voxel is delivered to a perception engine.

Early fusion of lidar return data with camera information

A machine vision system comprises a camera configured to generate one or more images of a field of regard of the camera, a lidar system, and a processor. The lidar system includes a laser configured to emit light, where the emitted light is directed toward a region within the field of regard of the camera and a receiver configured to detect light returned from the emitted light. The processor is configured to receive an indication of a location based on the returned light and determine whether a solid object is present at the location based on the one or more images.

Post-processing in a lidar system may be guided by camera information as described herein. In one embodiment, a camera system has a camera to capture images of the scene. An image processor is configured to classify an object in the images from the camera. A lidar system generates a point cloud of the scene and a modeling processor is configured to correlate the classified object to a plurality of points of the point cloud and to model the plurality of points as the classified object over time in a 3D model of the scene.

Post-processing by lidar system guided by camera information

A thermal imager is used to confirm object classifications as described. In one example, a scene modeling system has a sensor system to generate an image of a scene. A thermal camera generates a thermal image of the scene within a field of regard, and a modeling processor coupled to the sensor system and to the thermal camera correlates a position of a selected object in the scene to the field of regard of the thermal camera and queries the thermal camera to confirm a classification of the selected object. The thermal camera is configured to receive the position of the selected object and to confirm the classification.

Sensor system augmented with thermal sensor object confirmation

A dynamic vision sensor (DVS) camera is described that directs scanning of a lidar. In one example, a DVS has an array of pixels, wherein each DVS pixel comprises a photodetector, and is configured to detect temporal contrast and generate an event in response. A lidar system has a light source configured to emit light, a scanner configured to direct the emitted light along a scan pattern contained within a field of regard of the lidar system, and a receiver configured to detect at least a portion of the emitted light scattered by one or more remote targets in a scene. A processor is coupled to the lidar system and to the dynamic vision sensor and receives the events, identifies a region of interest in the field of regard that corresponds to DVS pixels that generated the events, and adjusts a scan parameter of the lidar system in the region of interest.

Growing community of inventors

Oviedo, FL, United States of America

Richmond Hicks