Huawei Technologies Co., Limited

Bingbing Liu

Ehsan Nezhadarya

Yuan Ren

Ehsan Taghavi

Amirhosein Nabatchian

Ran Cheng

Yang Liu

Eduardo R Corral-Soto

Ryan Razani

A system and method for processing a 3D point cloud to generate a segmented point cloud in real time are disclosed, the method includes: receiving a sparse 3D point cloud captured by a detection and ranging sensor mounted to a vehicle, the 3D point cloud comprising a plurality of data points, each data point in the 3D point cloud having a set of coordinates in a coordinate system of the detection and ranging sensor; generating, from the 3D point cloud, a range map comprising a plurality of elements, each of the plurality of data points of the 3D point cloud occupying a respective element of the plurality of elements; labelling the data point in each respective element of the range map as one of a pole-like data point or a vertical-plane-like data point; and generating the segmented point cloud including one or more of the labeled data points.

Method and system for real-time landmark extraction from a sparse three-dimensional point cloud

In methods and systems for graph-based panoptic segmentation of point clouds, points of a point cloud are received with a semantic label from a first category. Further, a plurality of unified cluster feature vectors from a second category are received, each being extracted from a cluster of points in the point cloud. Nodes of a constructed graph represent the unified feature vectors, and edges indicate the relationship between pairs of nodes. The edges are represented as an adjacency matrix indicating the existence or absence of an edge between pairs of nodes. A graph convolutional neural network uses the graph to predict an instance label for each node or an attribute for each edge, wherein the attribute of each edge is used for assigning the instance label to each node.

Method and system for graph-based panoptic segmentation

Methods and systems for deep neural networks using dynamically selected feature-relevant points from a point cloud are described. A plurality of multidimensional feature vectors arranged in a point-feature matrix are received. Each row of the point-feature matrix corresponds to a respective one of the multidimensional feature vectors, and each column of the point-feature matrix corresponds to a respective feature. Each multidimensional feature vector represents a respective unordered point from a point cloud and each multidimensional feature vector includes a respective plurality of feature-correlated values, each feature-correlated value represents a correlation extent of the respective feature. A reduced-max matrix having a selected plurality of feature-relevant vectors is generated. The feature-relevant vectors are selected by, for each respective feature, identifying a respective multidimensional feature vector in the point-feature matrix having a maximum feature-correlated value associated with the respective feature. The reduced-max matrix is output to at least one neural network layer.

Method and system for deep neural networks using dynamically selected feature-relevant points from a point cloud

A system and method for generating a bounding box for an object in proximity to a vehicle are disclosed. The method includes: receiving a three-dimensional (3D) point cloud representative of an environment; receiving a two-dimensional (2D) image of the environment; processing the 3D point cloud to identify an object cluster of 3D data points for a 3D object in the 3D point cloud; processing the 2D image to detect a 2D object in the 2D image and generate information regarding the 2D object from the 2D image; and when the 3D object and the 2D object correspond to the same object in the environment: generating a bird's eye view (BEV) bounding box for the object based on the object cluster of 3D data points and the information from the 2D image.

Method and system for generating a bird's eye view bounding box associated with an object

Devices, systems, methods, and media are described for adaptive scene augmentation of a point cloud frame for inclusion in a labeled point cloud dataset used for training a machine learned model for a prediction task for point cloud frames, such as object detection or segmentation. A formal method is described for generating new point cloud frames based on pre-existing annotated large-scale labeled point cloud frames included in a point cloud dataset to generate new, augmented point cloud frames. A policy is generated for large-scale data augmentation using detailed quantitative metrics such as confusion matrices. The policy is a detailed and stepwise set of rules, procedures, and/or conditions that may be used to generate augmented data specifically targeted to mitigate the existing inaccuracies in the trained model. The augmented point cloud frames may then be used to further train the model to improve the prediction accuracy of the model.

Devices, systems, methods, and media for adaptive augmentation for a point cloud dataset used for training

A system and method for localization of an autonomous vehicle based on 3D point cloud are disclosed. The system includes a processor and a memory coupled to the processor, the memory tangibly storing thereon executable instructions that, when executed by the processor, cause the processor to: receive an initial position of a vehicle; receive a data file representative of a three-dimensional (3D) point cloud from a LIDAR scanning subsystem of the vehicle; divide the 3D point cloud into a plurality of tiles and compute a plurality of key points for each tile; generate an estimated low frequency 2D position and yaw angle for the vehicle based on a particle filter process and a dynamically downloaded 2D reference map; and generate a position and orientation of the vehicle.

Method and system for localization of an autonomous vehicle in real time

Methods and apparatuses for generating a frame of semantically labeled 2D data are described. A frame of sparse 3D data is generated from a frame of sparse 3D data. Semantic labels are assigned to the frame of dense 3D data, based on a set of 3D bounding boxes determined for the frame of sparse 3D data. Semantic labels are assigned to a corresponding frame of 2D data based on a mapping between the frame of sparse 3D data and the frame of 2D data. The mapping is used to map a 3D data point in the frame of dense 3D data to a mapped 2D data point in the frame of 2D data. The semantic label assigned to the 3D data point is assigned to the mapped 2D data point. The frame of semantically labeled 2D data, including the assigned semantic labels, is outputted.

Method and system for semantic label generation using sparse 3D data

Upon receiving a set of two-dimensional data points representing an object in an environment, a bounding box estimator estimates a bounding box vector representative of a two-dimensional version of the object that is represented by the two-dimensional data points.

Estimating two-dimensional object bounding box information based on bird's-eye view point cloud

Methods and systems for encoding 3D data for use with 2D convolutional neural networks (CNNs) are described. A set of 3D data is encoded into a set of one or more arrays. A 2D index of the arrays is calculated by projecting 3D coordinates of the 3D point onto a 2D image plane that is defined by a set of defined virtual camera parameters. The virtual camera parameters include a camera projection matrix defining the 2D image plane. Each 3D coordinate of the point is stored in the arrays at the calculated 2D index. The set of encoded arrays is provided for input to a 2D CNN, for training or inference.

Growing community of inventors

Singapore, Singapore

Bingbing Liu