Nvidia Corporation

Deere + Company

David Wehr

Joachim Pehserl

Ibrahim Eden

Ke Chen

David Nister

Nikolai Smolyanskiy

Ruchi Bhargava

Alexander A Popov

Tilman Wekel

Ryan Oldja

Jack C Miller

Rafael Radkowski

Alex Renner

Nathan Sepich

Vinodh Sankaranthi

Chris Gallant

Cheng Song

Eliot Winer

Stephen Gilbert

Amir Akbarzadeh

In various examples, perception of landmark shapes may be used for localization in autonomous systems and applications. In some embodiments, a deep neural network (DNN) is used to generate (e.g., per-point) classifications of measured 3D points (e.g., classified LiDAR points), and a representation of the shape of one or more detected landmarks is regressed from the classifications. For each of one or more classes, the classification data may be thresholded to generate a binary mask and/or dilated to generate a densified representation, and the resulting (e.g., dilated, binary) mask may be clustered into connected components that are iteratively: fitted a shape (e.g., a polynomial or Bezier spline for lane lines, a circle for top-down representations of poles or traffic lights), weighted, and merged. As such, the resulting connected components and their fitted shapes may be used to represent detected landmarks and used for localization, navigation, and/or other uses.

Landmark perception for localization in autonomous systems and applications

A deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment. For example, a multi-view perception DNN may include multiple constituent DNNs or stages chained together that sequentially process different views of the 3D environment. An example DNN may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and/or regresses instance geometry in a second view (e.g., top-down). The DNN outputs may be processed to generate 2D and/or 3D bounding boxes and class labels for detected objects in the 3D environment. As such, the techniques described herein may be used to detect and classify animate objects and/or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.

Top-down object detection from LiDAR point clouds

Segmentation of lidar range images

In various examples, systems and methods are described that generate scene flow in 3D space through simplifying the 3D LiDAR data to '2.5D' optical flow space (e.g., x, y, and depth flow). For example, LiDAR range images may be used to generate 2.5D representations of depth flow information between frames of LiDAR data, and two or more range images may be compared to generate depth flow information, and messages may be passed—e.g., using a belief propagation algorithm—to update pixel values in the 2.5D representation. The resulting images may then be used to generate 2.5D motion vectors, and the 2.5D motion vectors may be converted back to 3D space to generate a 3D scene flow representation of an environment around an autonomous machine.

Belief propagation for range image mapping in autonomous machine applications

Methods, apparatus, systems and articles of manufacture are disclosed for augmented reality measuring of equipment. An example apparatus disclosed herein includes an image comparator to compare camera data with reference information of a reference vehicle part to identify a vehicle part included in the camera data, and an inspection image analyzer to, in response to the image comparator identifying the vehicle part, measure the vehicle part by causing an interface generator to generate an overlay representation of the reference vehicle part on the camera data displayed on a user interface, and determining, based on one or more user inputs to adjust the overlay representation, a measurement corresponding to the vehicle part.

Apparatus and methods for augmented reality measuring of equipment

Methods, apparatus, systems and articles of manufacture are disclosed for augmented reality vehicle condition inspection. An example apparatus disclosed herein includes a location analyzer to determine whether a camera is at an inspection location and directed towards a first vehicle in an inspection profile, the inspection location corresponding to a location of the camera relative to the first vehicle, an interface generator to generate an indication on a display that the camera is at the inspection location, the indication associated with an inspection image being captured, and an image analyzer to compare the inspection image captured at the inspection location with a reference image taken of a reference vehicle of a same type as the first vehicle, and determine a vehicle part condition or a vehicle condition based on the comparison of the inspection image and the reference image.

Apparatus and methods for augmented reality vehicle condition inspection

Multi-view deep neural network for LiDAR perception

Growing community of inventors

Redmond, WA, United States of America

David Wehr