Argo Ai Gmbh

Ford Global Technolgoies, LLC

Caterpillar Inc.

Carnegie Mellon University

Volkswagen Group of America Investments, LLC

Michael Happold

Xiaoyan Hu

Jason Ziglar

Lingyuan Wang

Guy Hotson

Joshua Max Manela

Cho-Ying Wu

Hunter Goforth

Ryan J Skaff

Derek Hartl

Anthony Joseph Stentz

William L Whittaker

Mark Ollis

Kerien W Fitzpatrick

Thomas E Pilarski

Nicolas Vandapel

Henning M Pangels

Methods and systems for jointly estimating a pose and a shape of an object perceived by an autonomous vehicle are described. The system includes data and program code collectively defining a neural network which has been trained to jointly estimate a pose and a shape of a plurality of objects from incomplete point cloud data. The neural network includes a trained shared encoder neural network, a trained pose decoder neural network, and a trained shape decoder neural network. The method includes receiving an incomplete point cloud representation of an object, inputting the point cloud data into the trained shared encoder, outputting a code representative of the point cloud data. The method also includes generating an estimated pose and shape of the object based on the code. The pose includes at least a heading or a translation and the shape includes a denser point cloud representation of the object.

Methods and systems for joint pose and shape estimation of objects from sensor data

Systems and methods for complementary control of an autonomous vehicle are disclosed. A primary controller provides a first plurality of instructions to an AV platform for operating the AV in an autonomous mode along a planned path based on sensor data from a primary sensor system and a secondary sensor system, and provides information that includes a fallback monitoring region to a complementary controller. The complementary controller receives sensor data from the secondary sensor system that includes sensed data for a fallback monitoring region, analyzes the received sensor data to determine whether a collision is imminent with an object detected in the fallback monitoring region, and cause the AV platform to initiate a collision mitigation action if a collision is determined to be imminent.

Complementary control system detecting imminent collision of autonomous vehicle in fallback monitoring region

Systems and methods for object detection. The methods include, by a computing device: obtaining a plurality of intensity values denoting at least a difference in a first location of at least one object in a first image and a second location of the at least one object in a second image; converting the intensity values to 3D position values; inputting the 3D position values into a classifier algorithm to obtain classifications for data points of a 3D point cloud (each of the classifications comprising a foreground classification or a background classification); and using the classifications to detect at least one object which is located in a foreground or a background.

Systems and methods for object detection using stereovision information

A system detects multiple instances of an object in a digital image by receiving a two-dimensional (2D) image that includes a plurality of instances of an object in an environment. For example, the system may receive the 2D image from a camera or other sensing modality of an autonomous vehicle (AV). The system uses a first object detection network to generate a plurality of predicted object instances in the image. The system then receives a data set that comprises depth information corresponding to the plurality of instances of the object in the environment. The data set may be received, for example, from a stereo camera of an AV, and the depth information may be in the form of a disparity map. The system may use the depth information to identify an individual instance from the plurality of predicted object instances in the image.

Geometry-aware instance segmentation in stereo image capture processes

A method and a system for generating a mesh representation of a surface. The method includes receiving a three-dimensional (3D) point cloud representing the surface, generating a reconstruction dataset having a higher resolution than the 3D point cloud in one or more regions corresponding to the surface from the 3D point cloud, and generate a polygon mesh representation of the surface by using a fine-to-coarse hash map for building polygons at a highest resolution first followed by progressively coarser resolution polygons, using the reconstruction dataset.

Methods and systems for constructing map data using poisson surface reconstruction

Systems and methods for object detection. The methods comprise, by a computing device: obtaining a plurality of intensity values denoting at least a difference in a first location of at least one object in a first image and a second location of the at least one object in a second image; converting the intensity values to 3D position values; inputting the 3D position values into a classifier algorithm to obtain classifications for data points of a 3D point cloud (each of the classifications comprising a foreground classification or a background classification); and using the classifications to detect at least one object which is located in a foreground or a background.

Systems and methods for operating a vehicle. The methods comprise, by a processor: obtaining a pair of stereo images captured by a stereo camera; processing the pair of stereo images to generate a disparity map comprising a plurality of pixels defined by intensity values; converting each intensity value to a 3D position in a map (each 3D position defining a location of a data point in a point cloud); performing a hierarchical decision tree classification to determine a classification for each data point in the point cloud (the classification being a foreground classification or a background classification); and using the classifications to facilitate autonomous control of the vehicle.

A method and a system for generating a mesh representation of a surface. The method includes receiving a three-dimensional (3D) point cloud representing the surface, identifying and discarding one or more outliers in the 3D point cloud to generate a filtered point cloud using a Gaussian process, adding one or more additional points to the filtered point cloud to generate a reconstruction dataset, and using Poisson surface reconstruction to generate an implicit surface corresponding to the surface from the reconstruction dataset.

A short range peer-to-peer communications system includes a first perception sensor and a first peer-to-peer communications system on a first machine and a second perception sensor and a second peer-to-peer communications system on a second machine. A first controller generates a first electronic map of a first perceived work environment associated with the first machine and transmits first data associated with first electronic map to the second machine through the first peer-to-peer communications system. A second controller generates a second electronic map of the second perceived work environment associated with the second machine based upon the plurality of second raw data points, receives the first data from the first machine through the second peer-to-peer communications system, and supplements the second electronic map based upon the first data.

Work site perception system

An agricultural harvesting machine (harvester) is automatically controlled and steered so as to completely harvest all crop in a field. A Global Positioning System (GPS) and an Inertial Navigation System provide data regarding the position and orientation of the harvester in a field. A Field Coverage Planner constructs a field coverage plan which defines a path for the harvester to follow in cutting crop from a field. Video images of the field in front of the harvester are derived alternately from two video cameras mounted on opposite sides of the harvester. The video images are processed in a Video Processing Computer to detect the crop line between cut and uncut crop and generate steering signals for steering the harvester along the crop line. Video images are also processed to detect the end of a row at the end of a field and supply a measure of the distance to the end of the row. When the end of a row is reached, the harvester executes a 180&deg; spin turn and traverses the field in the opposite direction.

Growing community of inventors

Pittsburgh, PA, United States of America

Michael Happold