Argo Ai Gmbh

Microsoft Technology Licensing, LLC

Ford Global Technolgoies, LLC

Uatc, LLC

Volkswagen Group of America Investments, LLC

Xiaoyan Hu

Michael Happold

Jason Ziglar

John Weiss

Lingyuan Wang

Guy Hotson

Hunter Goforth

Joshua Max Manela

Cho-Ying Wu

Rajesh M Dighde

Ying Zheng

Vivek Pradeep

Baoan Liu

Methods and systems for jointly estimating a pose and a shape of an object perceived by an autonomous vehicle are described. The system includes data and program code collectively defining a neural network which has been trained to jointly estimate a pose and a shape of a plurality of objects from incomplete point cloud data. The neural network includes a trained shared encoder neural network, a trained pose decoder neural network, and a trained shape decoder neural network. The method includes receiving an incomplete point cloud representation of an object, inputting the point cloud data into the trained shared encoder, outputting a code representative of the point cloud data. The method also includes generating an estimated pose and shape of the object based on the code. The pose includes at least a heading or a translation and the shape includes a denser point cloud representation of the object.

Methods and systems for joint pose and shape estimation of objects from sensor data

Systems and methods for object detection. The methods include, by a computing device: obtaining a plurality of intensity values denoting at least a difference in a first location of at least one object in a first image and a second location of the at least one object in a second image; converting the intensity values to 3D position values; inputting the 3D position values into a classifier algorithm to obtain classifications for data points of a 3D point cloud (each of the classifications comprising a foreground classification or a background classification); and using the classifications to detect at least one object which is located in a foreground or a background.

Systems and methods for object detection using stereovision information

A system detects multiple instances of an object in a digital image by receiving a two-dimensional (2D) image that includes a plurality of instances of an object in an environment. For example, the system may receive the 2D image from a camera or other sensing modality of an autonomous vehicle (AV). The system uses a first object detection network to generate a plurality of predicted object instances in the image. The system then receives a data set that comprises depth information corresponding to the plurality of instances of the object in the environment. The data set may be received, for example, from a stereo camera of an AV, and the depth information may be in the form of a disparity map. The system may use the depth information to identify an individual instance from the plurality of predicted object instances in the image.

Geometry-aware instance segmentation in stereo image capture processes

A method and a system for generating a mesh representation of a surface. The method includes receiving a three-dimensional (3D) point cloud representing the surface, generating a reconstruction dataset having a higher resolution than the 3D point cloud in one or more regions corresponding to the surface from the 3D point cloud, and generate a polygon mesh representation of the surface by using a fine-to-coarse hash map for building polygons at a highest resolution first followed by progressively coarser resolution polygons, using the reconstruction dataset.

Methods and systems for constructing map data using poisson surface reconstruction

Systems and methods for object detection. The methods comprise, by a computing device: obtaining a plurality of intensity values denoting at least a difference in a first location of at least one object in a first image and a second location of the at least one object in a second image; converting the intensity values to 3D position values; inputting the 3D position values into a classifier algorithm to obtain classifications for data points of a 3D point cloud (each of the classifications comprising a foreground classification or a background classification); and using the classifications to detect at least one object which is located in a foreground or a background.

Systems and methods for operating a vehicle. The methods comprise, by a processor: obtaining a pair of stereo images captured by a stereo camera; processing the pair of stereo images to generate a disparity map comprising a plurality of pixels defined by intensity values; converting each intensity value to a 3D position in a map (each 3D position defining a location of a data point in a point cloud); performing a hierarchical decision tree classification to determine a classification for each data point in the point cloud (the classification being a foreground classification or a background classification); and using the classifications to facilitate autonomous control of the vehicle.

Aspects of the present disclosure involve a vehicle computer system comprising a computer-readable storage medium storing a set of instructions, and a method for online light detection and ranging (Lidar) intensity normalization. Consistent with some embodiments, the method may include accumulating point data output by a channel of a Lidar unit during operation of an autonomous or semi-autonomous vehicle. The accumulated point data includes raw intensity values that correspond to a particular surface type. The method further includes calculating a median intensity value based on the raw intensity values and generating an intensity normalization multiplier for the channel based on the median intensity value. The intensity normalization multiplier, when applied to the median intensity value, results in a reflectivity value that corresponds to the particular surface type. The method further includes applying the intensity normalization multiplier to the point data output by the channel to produce normalized intensity values.

Online lidar intensity normalization

A method and a system for generating a mesh representation of a surface. The method includes receiving a three-dimensional (3D) point cloud representing the surface, identifying and discarding one or more outliers in the 3D point cloud to generate a filtered point cloud using a Gaussian process, adding one or more additional points to the filtered point cloud to generate a reconstruction dataset, and using Poisson surface reconstruction to generate an implicit surface corresponding to the surface from the reconstruction dataset.

Methods and apparatus for capturing motion from a self-tracking device are disclosed. In embodiments, a device self-tracks motion of the device relative to a first reference frame while recording motion of a subject relative to a second reference frame, the second reference frame being a reference frame relative to the device. In the embodiments, the subject may be a real object or, alternately, the subject may be a virtual subject and a motion of the virtual object may be recorded relative to the second reference frame by associating a position offset relative to the device with the position of the virtual object in the recorded motion. The motion of the subject relative to the first reference frame may be determined from the tracked motion of the device relative to the first frame and the recorded motion of the subject relative to the second reference frame.

Motion capture from a mobile self-tracking device

A system and method for tap event location includes a device using a selection apparatus that provides accurate point locations. The device determines a 3-dimensional map of a scene in the view frustum of the device relative to a coordinate frame. The device receives an indication of the occurrence of a tap event comprising a contact of the selection apparatus with a subject, and determines the location of the tap event relative to the coordinate frame from the location of the selection apparatus. The location of the tap event may be used to determine a subject. Data associated with the subject may then be processed to provide effects in, or data about, the scene in the view frustum of the device. Embodiments include a selection apparatus that communicates occurrences of tap events to the device and includes features that allow the device to determine the location of the selection apparatus.

Tap event location with a selection apparatus

The description relates to maintaining display device image quality. One example can include a display that includes a set of multiple pixels. Individual pixels comprise multiple color light emitting diodes (LEDs). The example can also include a processor configured to convert image related data into frame renderings for driving the multiple pixels of the display. The example can further include a context-aware anti-degradation component configured to receive a notification from the processor of a change to an individual frame rendering and to adjust lumination intensity values of unchanged portions of the individual frame rendering without comparing the individual frame rendering to a previous individual frame rendering.

Growing community of inventors

Redmond, WA, United States of America

Xiaoyan Hu