Lyft, Inc.

Blue Vision Labs UK Limited

Luca Del Pero

Long Chen

Benjamin Alex Haines

Karim Tarek Mahmoud Elsayed Ahmed Shaban

Peter Ondruska

Ivan Katanic

Matej Hamas

Yerzhan Utkelbayev

Robert Kesten

David Tse-Zhou Lu

Christy Fernandez Cull

Holger H Rapp

Wolfgang Hess

Lorenzo Peppoloni

Tayyab Naseer

Filippo Brizzi

Michal Szabados

Daniel Sievers

Jonathan Mazur Dyer

Charles J Richey

Stepan Simsa

Examples disclosed herein may involve a computing system that is operable to (i) identify a set of vehicle trajectories that are associated with a segment of a road network, (ii) identify a first cluster of sampling points between the identified set of vehicle trajectories and a first sampling position along the segment, wherein the first cluster has an associated geospatial position and is inferred to be associated with one given lane of the segment, (iii) identify a subset of vehicle trajectories in the identified set that are inferred to be associated with the given lane between the first sampling position and a second sampling position along the segment, (iv) identify a second cluster of sampling points between the identified subset of vehicle trajectories and the second sampling position, wherein the second cluster has an associated geospatial position, and (v) determine a geospatial geometry of the given lane.

Systems and methods for generating lane data using vehicle trajectory sampling

Examples disclosed herein may involve a computing system that is operable to (i) generate a local map portion of a geographical environment based on sensor data captured by a device, wherein the local map portion comprises local map structure data generated using one or more map structure generation methods, (ii) determine a transformation of the local map structure data relative to existing map structure data of an existing map based on common visible features between the local map structure data and the existing map structure data, wherein the existing map structure data is aligned to a global coordinate system and is predetermined from a plurality of previously-generated map structure data, and (iii) determine a localization of the device within the global coordinate system using the determined transformation.

Localization based on multi-collect fusion

Examples disclosed herein may involve a computing system that is operable to (i) receive first data of one or more geographical environments from a first type of localization sensor, (ii) receive second data of the one or more geographical environments from a second type of localization sensor, (iii) determine constraints from the first data and the second data, (iv) determine shared pose data associated with both of the first data and the second data using the constraints determined from both the first data and the second data by determining one or more sequences of common poses between respective poses generated from each of the first and second data, wherein the shared pose data provides a common coordinate frame for the first data and the second data, and (v) generate a map of the one or more geographical environments using the determined shared pose data.

Map generation using two sources of sensor data

The present invention relates to the curation of map data. More particularly, the present invention relates to a method for preparing map data to present to a data curator for substantially optimal quality assurance. Further, the present invention relates to a tool for a data curator to verify map data. According to a first aspect, there is provided a method comprising: generating a plurality of interdependent map portions from a global map; determining, from the plurality of interdependent map portions, at least one interdependent map portion that requires validation; creating at least one group of interdependent map portions, the group of interdependent map portions comprising: the determined at least one interdependent map portion that requires validation; and at least one additional interdependent map portion; and outputting the at least one group of interdependent map portions for validation.

Processing map data for human quality check

Examples disclosed herein involve a computing system configured to (i) based on image data captured by a vehicle in an environment, obtain observations of a time-sequence of positions of an agent identified within the image data, (ii) generate a first updated time-sequence of positions of the agent by performing a first optimization operation that includes processing the observed time-sequence of positions by beginning with a position associated with an observation having the highest confidence in the time-sequence of observations and proceeding in a first direction, (iii) after generating the first updated time-sequence of positions, generate a second updated time-sequence of positions of the agent by performing a second optimization operation that includes processing the first updated time-sequence of positions in a second direction opposite the first direction, and (iv) derive a trajectory for the agent in the environment based on the second updated time-sequence of positions for the agent.

Offline optimization of sensor data for agent trajectories

The present invention provides a method of generating a robust global map using a plurality of limited field-of-view cameras to capture an environment. Provided is a method for generating a three-dimensional map comprising: receiving a plurality of sequential image data wherein each of the plurality of sequential image data comprises a plurality of sequential images, further wherein the plurality of sequential images is obtained by a plurality of limited field-of-view image sensors; determining a pose of each of the plurality of sequential images of each of the plurality of sequential image data; determining one or more overlapping poses using the determined poses of the sequential image data; selecting at least one set of images from the plurality of sequential images wherein each set of images are determined to have overlapping poses; and constructing one or more map portions derived from each of the at least one set of images.

Guided batching

Examples disclosed herein may involve a computing system that is operable to (i) receive image data captured from one or more devices, (ii) based on the image data, generate at least two batches of data corresponding to an area of a global map, wherein each batch of data comprises (a) a respective group of images from the received image data, and (b) one or more common images comprising one or more common visual features, (iii) generate a respective reconstruction of the area of the global map for each of the at least two batches of data, and (iv) fuse the respective reconstructions of the area of the global map using the one or more common visual features from the one or more common images.

Fusing map reconstructions generated from multiple batches of data

Examples disclosed herein involve a computing system configured to (i) obtain first image data captured by a first camera of a vehicle during a given period of operation of the vehicle, (ii) obtain second image data captured by a second camera of the vehicle during the given period of operation, (iii) based on the obtained first and second image data, determine (a) a candidate extrinsics transformation between the first camera and the second camera and (b) a candidate time offset between the first camera and the second camera, and (iv) based on (a) the candidate extrinsics transformation and (b) the candidate time offset, apply optimization to determine a combination of (a) an extrinsics transformation and (b) a time offset that minimizes a reprojection error in the first image data, where the reprojection error is defined based on a representation of at least one landmark that is included in both the first and second image data.

Camera to camera calibration

Embodiments provide a hardware-based mechanism to continuously calibrate a set of sensors on a vehicle while the vehicle is in motion or in use. Specifically, each sensor is mounted on a respective rigid mounting device placed on the vehicle. A secondary sensing device for calibration is also rigidly mounted on the mounting device such that the relative position between the secondary device and the camera sensor can remain unchanged. The secondary sensing devices can then be used to detect a relative distance and/or position of the same calibration target. The sensed data is then used to determine, via stereo triangulation, the relative positions of the two secondary sensing devices, which in turn indicates the relative positions of the sensors.

Systems and methods for calibration of sensors on a vehicle

Examples disclosed herein may involve a computing system that is operable to (i) generate first structure data from one or more first image data, wherein the first structure data comprises one or more visible features captured in the one or more first image data, (ii) generate further structure data from one or more further image data, wherein the further structure data comprises one or more visible features captured in the one or more further image data, (iii) determine pose constraints for the further structure data based on common visible features, (iv) determine a transformation of the further structure data relative to the first structure data using the determined pose constraints, and (v) generate combined structure data using the determined transformation to fuse the further structure data and the first structure data.

Multi-collect fusion

Examples disclosed herein may involve a computing system that is operable to (i) receive image data and corresponding secondary sensor data, (ii) generate a reconstruction of a map from the image data, wherein the reconstruction comprises sequential pose information, (iii) determine constraints from the secondary sensor data, and (iv) validate the reconstruction of the map by applying the determined constraints from the secondary sensor data to the determined sequential pose information from the reconstruction of the map and determining whether the sequential pose information fails to satisfy any of the constraints determined from the secondary sensor data.

Validating a map reconstruction using sensor data constraints

Examples disclosed herein may involve a computing system that is operable to (i) receive one or more images related to a global map having a plurality of overlapping map segments, wherein each of the plurality of overlapping map segments overlaps with one or more neighboring map segments, (ii) based on a preliminary location determination for the one or more images, identify at least a first overlapping map segment of the plurality of overlapping map segments that corresponds to the one or more images, (iii) generate a reconstruction of the first identified overlapping map segment based on the one or more images, and (iv) fuse the generated reconstruction of the first identified overlapping map segment together with the first identified overlapping map segment's one or more neighboring map segments based on overlapping map portions between the generated reconstruction and the first identified overlapping map segment's one or more neighboring map segments.

Growing community of inventors

London, United Kingdom

Luca Del Pero