Beijing Didi Infinity Technology and Development Co., Ltd.

Google Inc.

Beijing Voyager Technology Co., Ltd.

Didi Research America, LLC

Tingbo Hou

Yan Zhang

Matthias Grundmann

Adel Ahmadyan

Jianing Wei

Guomin Xiang

Xufeng Han

Liangkai Zhang

Bent Hagemark

Xiaozhi Qu

Yanke Wang

Artsiom Ablavatski

The present disclosure is directed to systems and methods for performing object detection and pose estimation in 3D from 2D images. Object detection can be performed by a machine-learned model configured to determine various object properties. Implementations according to the disclosure can use these properties to estimate object pose and size.

Systems and methods for object detection including pose and size estimation

The present disclosure is directed to systems and methods for generating synthetic training data using augmented reality (AR) techniques. For example, images of a scene can be used to generate a three-dimensional mapping of the scene. The three-dimensional mapping may be associated with the images to indicate locations for positioning a virtual object. Using an AR rendering engine, implementations can generate an and orientation. The augmented image can then be stored in a machine learning dataset and associated with a label based on aspects of the virtual object.

AR-assisted synthetic data generation for training machine learning models

Systems and methods are disclosed related to generating an interactive user interface that enables a user to move, rotate or otherwise edit 3D point cloud data in virtual 3D space to align or match point clouds captured from LiDAR scans prior to generation of a high definition map. A system may obtain point cloud data for two or more point clouds, render the point clouds for display in a user interface, then receive a user selection of one of the point clouds and commands from the user to move and/or rotate the selected point cloud. The system may adjust the displayed position of the selected point cloud relative to the other simultaneously displayed point cloud(s) in real time in response to the user commands, and store the adjusted point cloud position data for use in generating a new high definition map.

Interactive 3D point cloud matching

The present disclosure is related to systems and methods for positioning. The method includes obtaining estimated pose data of a subject. The method also includes generating a local map associated with the estimated pose data. The method also includes obtaining, based on the estimated pose data, a reference map. The method also includes correlating the local map and the reference map in a frequency domain. The method further includes determining, based on the estimated pose data and the correlation between the local map and the reference map in the frequency domain, target pose data of the subject.

Systems and methods for positioning

A method includes receiving a video comprising images representing an object, and determining, using a machine learning model, based on a first image of the images, and for each respective vertex of vertices of a bounding volume for the object, first two-dimensional (2D) coordinates of the respective vertex. The method also includes tracking, from the first image to a second image of the images, a position of each respective vertex along a plane underlying the bounding volume, and determining, for each respective vertex, second 2D coordinates of the respective vertex based on the position of the respective vertex along the plane. The method further includes determining, for each respective vertex, (i) first three-dimensional (3D) coordinates of the respective vertex based on the first 2D coordinates and (ii) second 3D coordinates of the respective vertex based on the second 2D coordinates.

Object pose estimation and tracking using machine learning

Systems and methods involving obtaining point cloud data from one or more sources corresponding to one or more zones within a real-world space, the point cloud data representing surface features of structures detected within the one or more zones; defining a plurality of first-level nodes based on the point cloud data, individual first-level nodes corresponding to obtained point cloud data corresponding to individual zones of the one or more zones; identifying connections between two or more first-level nodes, the connections between the two or more first-level nodes based on connections between the point cloud data for the zones corresponding to the two or more first-level nodes; and defining a plurality of second-level nodes, individual second-level nodes corresponding to aggregated subsets of first-level nodes for which connections are identified.

Systems and methods for generating, updating and enhancing large-scale high-precision 3D road maps and multi-level road graphs

Example embodiments allow for fast, efficient determination of bounding box vertices or other pose information for objects based on images of a scene that may contain the objects. An artificial neural network or other machine learning algorithm is used to generate, from an input image, a heat map and a number of pairs of displacement maps. The location of a peak within the heat map is then used to extract, from the displacement maps, the two-dimensional displacement, from the location of the peak within the image, of vertices of a bounding box that contains the object. This bounding box can then be used to determine the pose of the object within the scene. The artificial neural network can be configured to generate intermediate segmentation maps, coordinate maps, or other information about the shape of the object so as to improve the estimated bounding box.

Real-time pose estimation for unseen objects

Technologies are disclosed that determine when alternative routes are to be pre-generated for a vehicle. A route planning system receives a request for a navigation route for a vehicle. The navigation route is generated, where the navigation route includes navigation nodes. For a given navigation node, a corresponding weight and/or a probability of an exception event occurring are determined, and are used to determine whether to generate an alternate route configured to be utilized in response to a future detection by the vehicle of an exception event associated with the given node. The route planning system may then generate alternate routes accordingly, and may transmit the navigation route and an alternate route for a corresponding given node to the vehicle. The navigation route may be used to navigate the vehicle, and the alternate route may be used to navigate the vehicle if an exception event is detected.

Systems and methods for safe route planning for a vehicle

Technologies are disclosed that are used to control the transmission and processing of vehicle telemetry data. Routing data configured to be used to navigate the vehicle is accessed. Using the routing data, telemetry broadcast parameters are generated. The telemetry broadcast parameters are transmitted to the vehicle. Transmissions of telemetry data in accordance with the broadcast parameters are received from the vehicle. The received telemetry data is used in generating a simulated environment for a route planning learning engine, in simulating a vehicle, in validating routes and maps, and/or in performing vehicle diagnostics.

Systems and methods for vehicle telemetry

Systems and processes can reduce an amount of training data used to generate a machine learning model while maintaining or improving a resultant of the machine learning model. The amount of training data may be reduced by pre-processing the training data to normalize the training data. The training data may include images of portions of an elongated object, such as a road. Each of the images can be normalized by, for example, rotating each of the images such that the depicted roads are horizontal or otherwise share the same angle. By aligning disparate images of roads, it is possible to reduce the amount of training data and to increase the accuracy of the machine learning model. Further, the use of normalized images by the machine learning model enables a reduction in computing resources used to apply data to the machine learning model to, for example, identify lane markings within images.

Using image pre-processing to generate a machine learning model

Systems and methods are provided for calibrating light intensity. An exemplary method for light intensity calibration may comprise: obtaining a plurality of intensity distributions of reflected light from an area, wherein each of the intensity distributions is associated with a beam; determining a reference intensity distribution from the plurality of intensity distributions, wherein the reference intensity distribution is associated with a reference beam, the plurality of intensity distributions excluding the reference intensity distribution are non-reference intensity distributions, and the non-reference intensity distributions are each associated with a non-reference beam; and aligning each of the non-reference intensity distributions to the reference distribution to calibrate the non-reference intensity distributions against the reference distribution.

System and method for calibrating light intensity

Systems and processes can automatically identify lane markings within images through the use of a machine learning model. The machine learning model may use a reduced set of data and output an improved estimate of lane markings by applying normalized data or images to the machine learning model. Each image applied to the model can be normalized by, for example, rotating each of the images such that the depicted roads are horizontal or otherwise share the same angle. By aligning disparate images of roads, it is possible to reduce the amount of data applied to the model or to model generation, and to increase the accuracy of the machine learning model. Further, the use of normalized images by the machine learning model enables a reduction in computing resources used to apply data to the machine learning model to, for example, identify lane markings within images.

System for automated lane marking

Systems and processes can reduce or divide images of road networks into sub-images that depict straight or substantially straight sections of roads in the road networks. These sub-images or image segments can be normalized by, for example, rotating each of the sub-images such that the depicted roads are horizontal or otherwise share the same angle. By aligning disparate images of roads, it is possible to both reduce the amount of training data used to generate a machine learning model and to increase the accuracy of an automated lane marking or labelling system. Further, the use of normalized images by the machine learning model enables a reduction in computing resources used to perform automated lane marking processes while maintaining or improving accuracy of the lane marking processes.

Image pre-processing in a lane marking determination system

Systems and methods are disclosed related to generating interactive user interfaces that enable a user to alter 3D point cloud data and/or associated pose graph data generated from LiDAR scans prior to generation of a high definition map. A user may make selections in a 2D map representation with overlaid graph node indicators in order to alter graph connections, remove nodes, view corresponding 3D point clouds, and otherwise edit intermediate results from LiDAR scans in order to improve the quality of a high definition map subsequently generated from the user-manipulated data.

Growing community of inventors

Santa Clara, CA, United States of America

Tingbo Hou