Luminar Technologies, Inc.

Luminar, LLC

Pranav Maheshwari

Benjamin Englard

Vahid R Ramezani

Meseret R Gebre

Shubham C Khilari

Tomi P Maila

Akshay Rangesh

Siddhesh S Mhatre

Ismail El Houcheimi

Austin K Russell

Dmytro Trofymov

Gauri Gandhi

Rounak Mehta

István Peter Burbank

A system for detecting boundaries of lanes on a road is presented. The system includes an imaging system configured to produce a set of pixels associated with lane markings on a road. The system also includes one or more processors configured to detect boundaries of lanes on the road, including: receive, from the imaging system, the set of pixels associated with lane markings; partition the set of pixels into a plurality of groups, each of the plurality of groups associated with one or more control points; and generate a first spline that traverses the control points of the plurality of groups, the first spline describing a boundary of a lane on the road.

Lane detection and tracking techniques for imaging systems

A method for determining a scan pattern according to which a sensor equipped with a scanner scans a field of regard (FOR) is presented. The method comprises obtaining, by processing hardware, a plurality of objective functions, each of the objective functions specifying a cost for a respective property of the scan pattern, expressed in terms of one or more operational parameters of the scanner. The method further includes applying, by the processing hardware, an optimization scheme to the plurality of objective functions to generate the scan pattern. The method further includes scanning the FOR according to the generated scan pattern.

Generating scan patterns using cognitive lidar

A scanning imaging sensor is configured to sense an environment through which a vehicle is moving. A method for determining one or velocities associated with objects in the environment includes generating features from the first set of scan lines and the second set of scan lines, the two sets corresponding to two instances in time. The method further includes generating a collection of candidate velocities based on feature locations and time differences, the features selected pairwise with one from the first set and another from the second set. Furthermore, the method includes analyzing the distribution of candidate velocities, for example, by identifying one or more modes from the collection of the candidate velocities.

Velocity determination with a scanned lidar system

A sequence of images generated at respective times by one or more sensors configured to sense an environment through which objects are moving relative to the one or more sensors is received. A message passing graph having a multiplicity of layers associated with the sequence of images is constructed. A neural network supported by the message passing graph is trained. The training includes performing a pass through the message passing graph in a forward direction including by adding a new feature node based on a feature detection and a new edge node and performing a pass through the message passing graph in a backward direction, including by updating at least one edge node of the message passing graph. Multiple features are tracked through the sequence of images, including passing messages through the message passing graph.

Neural network for object detection and tracking

A method for detecting boundaries of lanes on a road is presented. The method comprises receiving, by one or more processors from an imaging system, a set of pixels associated with lane markings. The method further includes partitioning, by the one or more processors, the set of pixels into a plurality of groups. Each of the plurality of groups is associated with one or more control points. The method further includes generating, by the one or more processors, a spline that traverses the control points of the plurality of groups. The spline traversing the control points describes a boundary of a lane.

To generate a machine learning model for controlling autonomous vehicles, training sensor data is obtained from sensors associated with one or more vehicles, the sensor data indicative of physical conditions of an environment in which the one or more vehicles operate, and a machine learning (ML) model is trained using the training sensor data. The ML model generates parameters of the environment in response to input sensor data. A controller in an autonomous vehicle receives sensor data from one or more sensors operating in the autonomous vehicle, applies the received sensor data to the ML model to obtain parameters of an environment in which the autonomous vehicle operates, provides the generated parameters to a motion planner component to generate decisions for controlling the autonomous vehicle, and causes the autonomous vehicle to maneuver in accordance with the generated decisions.

Generating environmental parameters based on sensor data using machine learning

A method for tracking a lane on a road is presented. The method comprises receiving, by one or more processors from an imaging system, a set of pixels associated with lane markings. The method further includes generating, by the one or more processors, a predicted spline comprising (i) a first spline and (ii) a predicted extension of the first spline in a direction in which the imaging system is moving. The first spline describes a boundary of a lane and is generated based on the set of pixels. The predicted extension of the first spline is generated based at least in part on a curvature of at least a portion of the first spline.

A method for multi-object tracking includes receiving a sequence of images generated at respective times by one or more sensors configured to sense an environment through which objects are moving relative to the one or more sensors, and constructing a message passing graph in which each of a multiplicity of layers corresponds to a respective one in the sequence of images. The method also includes tracking multiple features through the sequence of images, including passing messages in a forward direction and a backward direction through the message passing graph to share information across time.

A computer-implemented method of determining relative velocity between a vehicle and an object. The method includes receiving sensor data generated by one or more sensors of the vehicle configured to sense an environment by following a scan pattern comprising component scan lines. The method includes obtaining, based on the sensor data, a point cloud frame. Additionally, the method includes identifying a first pixel and a second pixel that are co-located within a field of regard and overlap a point cloud object within the point cloud frame and calculating a difference between a depth associated with the first pixel and a depth associated with the second pixel. The method includes determining a relative velocity of the point cloud object by dividing the difference in depth data by a time difference between when the depth associated with the first pixel was sensed and the depth associated with the second pixel was sensed.

Determining relative velocity using co-located pixels

A computer-readable medium stores instructions executable by one or more processors to implement a self-driving control architecture for controlling an autonomous vehicle. A perception and prediction component receives sensor data, and generates (1) an observed occupancy grid indicating which cells are currently occupied in a two-dimensional representation of the environment, and (2) predicted occupancy grids indicating which cells are expected to be occupied later. A mapping component provides navigation data for guiding the vehicle toward a destination, and a cost map generation component is configured to generate, based on the observed occupancy grid, the predicted occupancy grid(s), and the navigation data, cost maps that each specify numerical values representing a cost, at a respective instance of time, of occupying certain cells in a two-dimensional representation of the environment. A motion planner generates a grid path through the environment based on the cost maps, and corresponding decisions for maneuvering the vehicle.

Controlling an autonomous vehicle using cost maps

A computer-implemented method of determining a relative velocity between a vehicle and an object. The method includes receiving sensor data generated by one or more sensors of the vehicle configured to sense an environment following a scan pattern. The method also includes obtaining, based on the sensor data, a point cloud frame. The point cloud frame comprises a plurality of points of depth data and a time at which the depth data was captured. Additionally, the method includes selecting two or more points of the scan pattern that overlap the object. The selected points are located on or near a two-dimensional surface corresponding to the object, and the depth data for two or more of the selected points are captured at different times. The method includes calculating the relative velocity between the vehicle and the object based on the depth data and capture times associated with the selected points.

Growing community of inventors

Palo Alto, CA, United States of America

Pranav Maheshwari