Gm Cruise Holdings LLC

Hariprasad Govardhanam

Chen Shen

Sofya Pugach

Matthew Koehler

Zachary Monteith

System, methods, and computer-readable media for processing real-world environments into feature vectors that can be randomly selected into a synthetic scene. Scene data is provided to a generative machine-learning network that processes road types, environment lighting conditions, object behaviors, and AV trajectories of an AV driving scene from the sensor data into feature embeddings that represent such features from the driving scenes. The feature embeddings may be stored as unique character profiles in a scene database to be randomly selected into the synthetic scene.

Generating simulations based on real-world scenarios

The disclosed technology provides solutions for improving object tracking. In some aspects, a process of the disclosed technology can include steps for receiving Light Detection and Ranging (LiDAR) point cloud data corresponding with an object, providing the LiDAR point cloud data to an autoencoder neural network to generate a set of feature embeddings, and tracking the object based on the feature embeddings. Systems and machine-readable media are also provided.

Object tracking using semantic attributes

System, methods, and computer-readable media for a training technique of an object trajectory prediction model that inputs semantic map data pertaining to an environment and trajectory data of an object into an autoencoder neural network, which outputs the original trajectory as a feature embedding vector. Each feature embedding vector serves as a unique identifier for each behavior. Cluster analysis is performed on feature embedding vectors to determine clusters, each associated with a particular behavior attribute.

Determining object behavior and trajectories

Aspects of the disclosed technology provide solutions for measuring divergence between recorded real-world scene data, and a simulated environment. A process of the disclosed technology can include steps for generating, based on real-world autonomous vehicle (AV) scene data, a computer-generated simulation of a real-world scenario, determining, an occlusion region in which at least one surrounding object obstructs a field of view of the AV, and determining, based on the occlusion region, a portion of the object of interest that is visible to the AV. In some aspects, the process can further include steps for determining a divergence value indicating a difference between the portion of the object of interest that is visible to the AV within the simulation of the real-world scenario to an actual portion of the object of interest that is visible to the AV within the real-world scenario. Systems and machine-readable media are also provided.

Measuring environmental divergence in a simulation using object occlusion estimation

The disclosed technology provides solutions for measuring sensor realism (or fidelity) and in particular, provides methods for quantitatively determining how various sensor modeling parameters contribute to an associated fidelity metric. In some aspects, a process of the disclosed technology includes steps for receiving a first set of sensor data, determining a first fidelity score for the first set of sensor data, and processing the first set of sensor data using a perturbation algorithm to generate a second set of sensor data. In some aspects, the process can further include steps for determining a second fidelity score for the second set of sensor data. Systems and machine-readable media are also provided.

Attributing sensor realism gaps to sensor modeling parameters

The disclosed technology provides solutions for generating synthetic 3D environments, In some aspects, the disclosed technology includes a process of collecting sensor data corresponding with a three-dimensional (3D) space; identifying objects in the sensor data; filtering objects for inclusion in the synthetic map based on (1) a corresponding influence score for each of the one or more of the objects and (2) a corresponding duration of presence of each of the objects in the sensor data; selecting one or more of the objects based on the filtering, wherein the corresponding influence score of each of the one or more of the objects is greater than a first threshold and the corresponding duration of presence of each of the one or more of the objects is greater than a second threshold; and generating the synthetic map including a 3D rendering of the one or more of the objects selected.

Filtering real-world objects for inclusion in a simulated environment

The disclosed technology provides solutions for generating synthetic scenes based on sensor data and in particular, for generating synthetic representations of entities using splines. A process of the disclosed technology can include steps for extracting trajectory data associated with movement of an entity in an environment, generating splines based on the trajectory data, and generating a synthetic scene based on the splines. Systems and machine-readable media are also provided.

Synthetic scene generation using spline representations of entity trajectories

The disclosed technology provides solutions for generating synthetic objects and in particular, for generating three-dimensional objects using known geometries that can be queried from a database using feature vector information. A process of the disclosed technology can include steps for receiving sensor data including a detected object, wherein the sensor data represents one or more three-dimensional (3D) attributes of the detected object, processing the sensor data to identify at least one intrinsic attribute associated with the detected object, and querying an object database, based on the at least one intrinsic attribute, to identify a stored object that is similar to the detected object. In some aspects, the process can further include steps for modifying one or more attributes of the stored object based on the one or more 3D attributes of the detected object to generate a synthetic object. Systems and machine-readable media are also provided.

Generating synthetic three-dimensional objects

The disclosed technology provides solutions for vehicle routing and in particular, for facilitating obstacle avoidance maneuvering by an autonomous vehicle (AV). In some implementations, a process of the disclosed technology can include steps for collecting environmental data about an environment around an autonomous vehicle, wherein the environmental data comprises data pertaining to a roadway navigated by the autonomous vehicle and one or more other vehicles navigating the roadway, processing the environmental data to generate an area grid comprising one or more grid sections, wherein each grid section corresponds with a different area of the roadway, and determining a risk-profile for the one or more grid sections based on the environmental data. Systems and machine-readable media are also provided.

Obstacle detection based on other vehicle behavior

The disclosed technology provides solutions for improving passenger drop-off functions implemented by an autonomous vehicle (AV). In some implementations, a process of the disclosed technology can include steps for collecting environmental data about an environment around an autonomous vehicle, wherein the environmental data comprises data pertaining to a roadway navigated by the autonomous vehicle, processing the environmental data to generate an area grid comprising a plurality of grid sections, and associating, based on the environmental data, one or more features with each of the plurality of grid sections. Systems and machine-readable media are also provided.

Growing community of inventors

Fremont, CA, United States of America

Hariprasad Govardhanam