Nvidia Corporation

Ford Global Technolgoies, LLC

Argo Ai Gmbh

Volkswagen Group of America Investments, LLC

Fnu Ratnesh Kumar

Parthasarathy Sriram

Anil Ubale

Yan Zhai

Subhashree Radhakrishnan

James Hays

Edwin Weill

Ming-Fang Chang

De Wang

G Peter K Carr

Xiufeng Song

Kevin Lee Wyffels

Wulue Zhao

Minhan Li

This document discloses system, method, and computer program product embodiments for detecting an object. For example, the method includes generating a plurality of cuboids by performing the following operations: defining a plurality of first cuboids each encompassing lidar data points that are plotted on a respective 3D graph of a plurality of 3D graphs; accumulating the lidar data points encompassed by the plurality of first cuboids; computing an extent using the accumulated lidar data points; and defining a second cuboid that has dimensions specified by the extent. The first cuboids and/or the second cuboid may be used to detect the object.

Systems and methods for producing amodal cuboids

Disclosed herein are systems, methods, and computer program products for operating a robotic system. For example, the method includes: obtaining a first cuboid generated based on an image, a second cuboid generated based on a lidar dataset and/or a third cuboid generated by a heuristic algorithm using the lidar dataset; using a machine learning model to generate a heading for an object in proximity to the robotic system based on the first cuboid, second cuboid and/or third cuboid; generating a bounding box geometry and a bounding box location based on the second cuboid or third cuboid; and generating a fourth cuboid using the bounding box geometry, the bounding box location, and the heading generated using the machine learning model.

Systems and methods for estimating cuboid headings based on heading estimations generated using different cuboid defining techniques

Systems may include at least one processor configured to determine a predicted value of an unwrap factor using a machine learning model, wherein the machine learning model is a trained machine learning model configured to provide a predicted value of an unwrap factor for dealiasing a measurement of range rate of a target object as an output, dealiase a measurement value of range rate from a radar of an autonomous vehicle (AV) based on the predicted value of the unwrap factor to provide a true value of range rate, and control an operation of the AV in a real-time environment based on the true value of range rate. Methods, computer program products, and autonomous vehicles are also disclosed.

Methods and systems for dealiasing radar range rate measurements using machine learning

In various examples, a neural network may be trained for use in vehicle re-identification tasks—e.g., matching appearances and classifications of vehicles across frames—in a camera network. The neural network may be trained to learn an embedding space such that embeddings corresponding to vehicles of the same identify are projected closer to one another within the embedding space, as compared to vehicles representing different identities. To accurately and efficiently learn the embedding space, the neural network may be trained using a contrastive loss function or a triplet loss function. In addition, to further improve accuracy and efficiency, a sampling technique—referred to herein as batch sample—may be used to identify embeddings, during training, that are most meaningful for updating parameters of the neural network.

Training neural networks for vehicle re-identification

In various examples, sensor data—such as masked sensor data—may be used as input to a machine learning model to determine a confidence for object to person associations. The masked sensor data may focus the machine learning model on particular regions of the image that correspond to persons, objects, or some combination thereof. In some embodiments, coordinates corresponding to persons, objects, or combinations thereof, in addition to area ratios between various regions of the image corresponding to the persons, objects, or combinations thereof, may be used to further aid the machine learning model in focusing on important regions of the image for determining the object to person associations.

Determining associations between objects and persons using machine learning models

Methods of determining relevance of objects that a vehicle detected are disclosed. A system will receive a data log of a run of the vehicle. The data log includes perception data captured by vehicle sensors during the run. The system will identify an interaction time, along with a look-ahead lane based on a lane in which the vehicle traveled during the run. The system will define a region of interest (ROI) that includes a lane segment within the look-ahead lane. The system will identify, from the perception data, objects that the vehicle detected within the ROI during the run. For each object, the system will determine a detectability value by measuring an amount of the object that the vehicle detected. The system will create a subset with only objects having at least a threshold detectability value, and it will classify any such object as a priority relevant object.

Using relevance of objects to assess performance of an autonomous vehicle perception system

In various examples, sensor data&#x2014;such as masked sensor data&#x2014;may be used as input to a machine learning model to determine a confidence for object to person associations. The masked sensor data may focus the machine learning model on particular regions of the image that correspond to persons, objects, or some combination thereof. In some embodiments, coordinates corresponding to persons, objects, or combinations thereof, in addition to area ratios between various regions of the image corresponding to the persons, objects, or combinations thereof, may be used to further aid the machine learning model in focusing on important regions of the image for determining the object to person associations.

Systems and methods for operating an autonomous vehicle. The methods comprising: obtaining, by a computing device, loose-fit cuboids overlaid on 3D graphs so as to each encompass LiDAR data points associated with a given object; defining, by the computing device, an amodal cuboid based on the loose-fit cuboids; using, by the computing device, the amodal cuboid to train a machine learning algorithm to detect objects of a given class using sensor data generated by sensors of the autonomous vehicle or another vehicle; and causing, by the computing device, operations of the autonomous vehicle to be controlled using the machine learning algorithm.

In various examples, a neural network may be trained for use in vehicle re-identification tasks&#x2014;e.g., matching appearances and classifications of vehicles across frames&#x2014;in a camera network. The neural network may be trained to learn an embedding space such that embeddings corresponding to vehicles of the same identify are projected closer to one another within the embedding space, as compared to vehicles representing different identities. To accurately and efficiently learn the embedding space, the neural network may be trained using a contrastive loss function or a triplet loss function. In addition, to further improve accuracy and efficiency, a sampling technique&#x2014;referred to herein as batch sample&#x2014;may be used to identify embeddings, during training, that are most meaningful for updating parameters of the neural network.

Growing community of inventors

Campbell, CA, United States of America

Fnu Ratnesh Kumar