Scale Ai, Inc.

Leigh Marie Braswell

Evan Moss

Steven Hao

Chiao-Lun Cheng

Nathaniel John Herman

Samuel Jacob Clearman

Akshat Bubna

Guido Leandro Maliandi

Elliot Branson

Daniel Havíř

Jeffrey Alan Anders

One embodiment provides techniques for automatically pre-labeling point cloud data with cuboid annotations. Point cloud data is processed using ML models to detect, associate, and localize objects therein, in order to generate cuboid tracks that each include a series of cuboid annotations associated with an object. An object detection model that detects objects and performs coarse localization is trained using a loss function that separately evaluates the distances between corners of predicted cuboids and corners of ground truth cuboids for position, size, and yaw. A refinement model that performs more accurate localization takes as input 2D projections of regions surrounding cuboid tracks predicted by the object detection model and the cuboid tracks, and outputs refined cuboid tracks. The refined cuboid tracks are filtered to a set of keyframes, with in-between frames being interpolated. The cuboid tracks can then be presented to a user for viewing and editing.

Pre-labeling data with cuboid annotations

One embodiment provides a user interface (UI) that permits users to select how point cloud colorings determined from multiple data sources are blended together in a rendering of a point cloud. The data sources may include photographic, label, and/or LIDAR intensity data. To improve frame rates, an aggregated point cloud may be generated using a spatial hash of a large set of points and sampling of each hash bucket based on the number of points therein and a user-configurable density. Sizes of points in the point cloud may decrease proportionally to distance from a viewer, but increase based on an activation function that enlarges points greater than a threshold distance from the viewer. In addition, luminance statistics for sub-regions of photographic data and dominant colors determined from photographic data may be used to automatically determine color properties to apply to a point cloud coloring.

Visualization techniques for data labeling

Techniques for coloring a point cloud based on colors derived from LIDAR (light detection and ranging) intensity data are disclosed. In some embodiments, the coloring of the point cloud may employ an activation function that controls the colors assigned to different intensity values. Further, the activation function may be parameterized based on statistics computed for a distribution of intensities associated with a 3D scene and a user-selected sensitivity. Alternatively, a Fourier transform of the distribution of intensities or a clustering of the intensities may be used to estimate individual distributions associated with different materials, based on which the point cloud coloring may be determined from intensity data.

Intensity data visualization

Growing community of inventors

San Francisco, CA, United States of America

Leigh Marie Braswell