Here Global B.v.

Anirudh Viswanathan

Peter Christian

Matt Mott

Anish Mittal

Richard Kwant

Vishal Jain

Methods, systems, apparatuses, and computer program products are provided that are configured to processor sensor data using a single component specialized sensor data processing neural network. The single component data processing system is configured to receive input data from a sensor, analyze the input data using a neural network embodied by a single trained sensor data processing component, wherein the single trained sensor data processing component is trained to product output data that approximates a plurality of task-specific transformations performed in a pipeline manner, and produce the output data following transformation of the input data. Additionally, methods, systems, apparatuses, and computer program products are provided for training a single trained sensor data processing component to approximate a plurality of task-specific transformations performed in a pipeline manner.

Single component data processing system and method utilizing a trained neural network

A method, apparatus, and computer program product are provided to determine sensor orientation. In the context of a method, the orientation of a sensor is predicted based on a predetermined dataset. The method also includes receiving information regarding a location of the sensor and information from which a gravity direction of the sensor is derivable. The method further includes determining a relative orientation of the sensor in relation to a frame of reference and the gravity direction of the sensor. The method still further includes comparing the orientation of the sensor that was predicted and the relative orientation of the sensor to determine a prediction accuracy. The method finally includes updating the predetermined dataset based on the prediction accuracy of the sensor orientation.

Method, apparatus, and computer program product for determining sensor orientation

Methods described herein relate to reducing the computational intensity of vision-based localization. Methods may include: receiving sensor data from a vehicle traveling along a road; identifying one or more features of the environment from the sensor data; classifying the one or more identified features into one or more of a plurality of semantic classifications for the features; identifying map image data based on an identified location of the vehicle; identifying one or more features in the map image data; comparing one or more identified features of a first semantic classification with one or more features of the map image data of the first semantic classification; and registering a localized location of the vehicle within the environment based, at least in part, on the one or more identified features of the first semantic classification corresponding to the one or more features of the map image data of the first semantic classification.

Method and apparatus for localization using search space pruning

A method, apparatus and computer program product are provided for warping a perspective image into the ground plane using a homography transformation to estimate a bird's eye view in real time. Methods may include: receiving first sensor data from a first vehicle traveling along a road segment in an environment, where the first sensor data includes perspective image data of the environment, and where the first sensor data includes a location and a heading; retrieving a satellite image associated with the location and heading; applying a deep neural network to regress a bird's eye view image from the perspective image data; applying a Generative Adversarial Network (GAN) to the regressed bird's eye view image using the satellite image as a target of the GAN to obtain a stabilized bird's eye view image; and deriving values of a homography matrix between the sensor data and the established bird's eye view image.

Method, apparatus, and computer program product for generating an overhead view of an environment from a perspective image

A method, apparatus and computer program product are provided for establishing the reliability of crowd sourced data based on the source of the data and the region in which the data is gathered. Methods may include: receiving map data for a network of roads in a geographic area; receiving location accuracy data from different regions within the geographic area, where the location accuracy data for each region may be indicative of an accuracy with which location can be established in the respective region; receiving a plurality of probe data points; determining, for each probe data point, the location accuracy data associated with the location information associated with the respective probe apparatus; determining, for each probe apparatus, a reliability of the one or more sensors; and updating a map database based on at least one probe data point.

Method, apparatus, and computer program product for establishing reliability of crowd sourced data

An approach is provided for providing predictive classification of sensor error. The approach involves, for example, receiving sensor data from at least one sensor, the sensor data collected at a geographic location. The approach also involves extracting a set of input features from the sensor data, map data representing the geographic location, or combination thereof. The approach further involves processing the set of input features using a machine learning model to calculate a predicted sensor error of a target sensor operating at the geographic location. The machine learning model, for instance, has been trained on ground truth sensor error data to use the set of input features to calculate the predicted sensor error.

Method and apparatus for predicting sensor error

Various methods are provided for facilitating map update to an environment map using gradient thresholding. One example method may include detecting an observed feature associated with a first feature decay and generating an interpolated feature that approximates the observed feature associated with a second decay. The method also includes determining a gradient difference between the interpolated feature and a stored map feature. The stored map feature represents an encoding of the observed feature associated with a third decay associated with an environment map. The method also includes determining a relationship between the gradient difference and a feature gradient update threshold, and, based upon the relationship, updating the environment map by at least replacing the map feature representation associated with the environment map with the approximated feature representation.

Method and apparatus for updating maps using gradient thresholding

Described herein are methods of generating learning data to facilitate de-biasing the labeled location of an object of interest within an image. Methods may include: receiving sensor data, where the sensor data is a first image; determining reference corner locations of an object in the first image using image processing; generating observed corner locations of the object in the first image from the determined reference corner locations; generating a bias transformation based, at least in part, on a difference between the reference corner locations and the observed corner locations of the object in the first image; receiving sensor data from another image sensor of a second image; receiving observed corner locations of an object in the second image from a user; and applying the bias transformation to the observed corner locations of the object in the second image to generate de-biased corners for the object in the second image.

Method and apparatus for de-biasing the detection and labeling of objects of interest in an environment

Described herein are methods of detecting and labeling features within an image of an environment. Methods may include: receiving sensor data from an image sensor, where the sensor data is representative of a first image including an aerial view of a geographic region; detecting, using a perception module, at least one vehicle within the image of the geographic region; identifying an area around the at least one vehicle as a road segment in response to detecting the at least one vehicle; based on the identification of the area around the vehicle as a road segment, identifying features within the area as road features based on a context of the area; generating a map update for the road features of the road segment; and causing a map database to be updated with the road features of the road segment.

Method and apparatus for the detection and labeling of features of an environment through contextual clues

An approach is provided for fully-automated learning to match heterogeneous feature spaces for mapping. The approach involves determining a first feature space comprising first features and a second feature space comprising second features, and classified by a feature detector into a first attribution category and a second attribution category, respectively. The approach further involves calculating a first similarity score for the first feature space based on a first distance metric applied to the first features, and a second similarity score for the second feature space based on a second distance metric applied to the second features. The approach also involves determining a transformation space comprising a first weight to be applied to the first similarity score and a second weight to be applied to the second similarity score based on matching the first attribution category and the second attribution category.

Method and apparatus for matching heterogeneous feature spaces

Methods, systems, apparatuses, and computer program products are provided that are configured to perform localization of position data, specifically using a trained localization neural network. In the context of an apparatus, the apparatus is caused to receive observed feature representation data. The apparatus is further configured to transform the observed feature representation data into standardized feature representation data utilizing a trained localization neural network. The apparatus is further configured to compare the standardized feature representation data and the map feature representation data and identify local position data based on the comparison.

Method and apparatus for localization of position data

Methods described herein relate to iteratively refining parameters of a localization framework for different data sources. Methods may include: receiving a map data set from a data source; applying an offset to the map data to generate an offset map data set; applying a localization framework between the map data set and the offset map data set to generate a localized data set; recovering offset data from the localized data set; iteratively tuning parameters of the localization framework in response to the offset data from the localized data set failing to correspond with the applied offset until offset data recovered from the localized data set corresponds to the applied offset; receiving sensor data from at least one sensor of a vehicle; applying the localization framework to the sensor data to locate the vehicle within a mapped region.

Method and apparatus for iterative refinement of parameters of a localization framework

A method is provided for dynamically throttling processing and memory consumption of sensors based on context. Methods may include: receiving first sensor data from a first sensor, where the first sensor data includes data associated with an environment of the sensor; applying an auto-encoder framework to the first sensor data to establish a data difference score between frames of the first sensor data, where a relatively high data difference score corresponds to substantial differences between frames of the first sensor data, and wherein a relatively low data difference score corresponds to insubstantial differences between frames of the first sensor data; reducing a frame rate of data capture of the first sensor in response to the data difference score between frames of the first sensor data being relatively low; capturing first sensor data from the first sensor at the reduced frame rate; and providing for storage of the first sensor data.

Method and apparatus for regulating resource consumption by one or more sensors of a sensor array

A method is provided for generating training data to facilitate automatically locating an object of interest within an image. Methods may include: receiving sensor data including a plurality of images from at least one image sensor; receiving an identification, from a user, of an object visible within an image of the plurality of images, where at least a portion of the object is visible in one or more of the plurality of images; determining a predicted location of the object in the one or more of the remaining images of the plurality of images; identifying the object in the one or more of the remaining images of the plurality of images; and storing the plurality of images including an indication of the object at the object location within the one or more of the plurality of images.

Method and apparatus for detecting objects of interest in an environment

An approach is provided for storing and retrieving map data using contextual information priors. The approach involves, for example, processing contextual information to determine a restricted range of location information relevant to at least one query. The approach also involves processing sensor data received from at least one sensor, the sensor data collected at at least one query location, to determine semantic information. The approach further involves filtering the map data based, at least in part, on the restricted range of location information relevant to the at least one query, the semantic information, or a combination thereof.

Growing community of inventors

Berkeley, CA, United States of America

Anirudh Viswanathan