Honda Motor Co., Ltd.

Bernd Heisele

Alper Ayvaci

Kai-Chi Chan

Yoav Sharon

Hiroshi Tsujino

Yi-Ting Chen

Takamasa Koshizen

Yoshiaki Sakagami

Christian Goerick

Jiawei Huang

Yan Lu

Vasiliy Igorevich Karasev

Andrew J Meyer

Yingying Zhu

Ali Taalimi

Jan Fritsch

Jingjing Zheng

Gunhee Kim

The present disclosure relates to methods and systems for monocular localization in urban environments. The method may generate an image from a camera at a pose. The method may receive a pre-generated map, and determine features from the generated image based on edge detection. The method may predict a pose of the camera based on at least the pre-generated map, and determine features from the predicted camera pose. Further, the method may determine a Chamfer distance based upon the determined features from the image and the predicted camera pose, optimize the determined Chamfer distance based upon odometry information and epipolar geometry. Upon optimization, the method may determine an estimated camera pose.

Monocular localization in urban environments using road markings

A system and method for providing laser camera fusion for identifying and tracking a traffic participant that include receiving an image of a surrounding environment of a vehicle and a set of object coordinates of at least one object determined within the surrounding environment of the vehicle. The system and method also include determining a portion of the image as object space based on the image and the set of object coordinates and filtering the object space to identify a traffic related object. Additionally, the system and method include determining a three dimensional position of the traffic related object and classifying the traffic related object as at least one of: the traffic participant, or a non-traffic participant. The system and method further include tracking the traffic participant based on a three dimensional position of the traffic related object classified as the traffic participant.

System and method for providing laser camera fusion for identifying and tracking a traffic participant

A method for partially occluded object detection includes obtaining a response map for a detection window of an input image, the response map based on a trained model and including a root layer and a parts layer. The method includes determining visibility flags for each root cell of the root layer and each part of the parts layer. The visibility flag is one of visible or occluded. The method includes determining an occlusion penalty for each root cell with a visibility flag of occluded and for each part with a visibility flag of occluded. The occlusion penalty is based on a location of the root cell or the part with respect to the detection window. The method determines a detection score for the detection window based on the visibility flags and the occlusion penalties and generates an estimated visibility map for object detection based on the detection score.

System and method for partially occluded object detection

A system and method for verifying detection of partially occluded objects (e.g., pedestrians) in the vicinity of a vehicle. An image input device captures an image and/or video of surroundings. An object detector detects partially occluded pedestrians and other objects in received image information. The detection of a partially occluded pedestrian is verified when a detection window of the partially occluded pedestrian overlaps with a detection window of an occluding object, and the occluding object window is closer to the image input device than the partially occluded object window. Optionally, a range-finding sensor, such as a LIDAR device, determines a range to objects located in the direction of the partially occluded object. The detection of the partially occluded object is verified when the range of one of the other objects located in the direction of the partially occluded object is less than that of the partially occluded object.

Partially occluded object detection using context and depth ordering

Systems and techniques for pedestrian path predictions are disclosed herein. For example, an environment, features of the environment, and pedestrians within the environment may be identified. Models for the pedestrians may be generated based on features of the environment. A model may be indicative of goals of a corresponding pedestrian and predicted paths for the corresponding pedestrian. Pedestrian path predictions for the pedestrians may be determined based on corresponding predicted paths. A pedestrian path prediction may be indicative of a probability that the corresponding pedestrian will travel a corresponding predicted path. Pedestrian path predictions may be rendered for the predicted paths, such as using different colors or different display aspects, thereby enabling a driver of a vehicle to be presented with information indicative of where a pedestrian is likely to travel.

Pedestrian path predictions

Methods and systems for determining a vehicle location and pose are provided. A vehicle localization system may determine a location and pose estimate for the vehicle. The system may retrieve three-dimensional map data based on the location and pose estimate. The system may obtain a camera image from a vehicle mounted camera. The system may project the three-dimensional map data onto the camera image based on the location and pose estimate. The system may compare the projected three-dimensional map data to the camera image. The system may then determine the vehicle location and pose based on the comparing.

System and method for image based vehicle localization

A pedestrian pose classification model is trained. A three-dimensional (3D) model of a pedestrian is received. A set of image parameters indicating how to generate an image of a pedestrian is received. A two-dimensional (2D) synthetic image is generated based on the received 3D model and the received set of image parameters. The generated synthetic image is annotated with the set of image parameters. A plurality of pedestrian pose classifiers is trained through the annotated synthetic image.

3D human models applied to pedestrian pose classification

A determination is made in real-time regarding whether a bicyclist is present in a target image. A target image is received. The target image is classified and an error value for the target image is determined using a linear classifier. If the error value does not exceed the threshold value, the classification is outputted. Otherwise, if the error value exceeds the threshold value, the target image is classified using a non-linear classifier.

Real-time bicyclist detection with synthetic training data

A vehicle periphery monitoring deviceis equipped with a first recognizing unitwhich recognizes a shape of a road in a traveling direction of a self vehicle C, a second recognizing unitwhich recognizes a position of an object M existing in a periphery of the self vehicle C, an estimating unitwhich estimates a moving direction of the object M from the position recognized by the second recognizing unit, and a contact avoiding process unitwhich determines whether or not to perform an avoiding process for avoiding contact between the self vehicle C and the object M, on the basis of a relationship between the shape of the road recognized by the first recognizing unitand the moving direction of the object M estimated by the estimating unit

Vehicle periphery monitoring device

Object recognition techniques are disclosed that provide both accuracy and speed. One embodiment of the present invention is an identification system. The system is capable of locating objects in images by searching for local features of an object. The system can operate in real-time. The system is trained from a set of images of an object or objects. The system computes interest points in the training images, and then extracts local image features (tokens) around these interest points. The set of tokens from the training images is then used to build a hierarchical model structure. During identification/detection, the system computes interest points from incoming target images. The system matches tokens around these interest points with the tokens in the hierarchical model. Each successfully matched image token votes for an object hypothesis at a certain scale, location, and orientation in the target image. Object hypotheses that receive insufficient votes are rejected.

Using a model tree of group tokens to identify an object in an image

An 'active learning' method trains a compact classifier for view-based object recognition. The method actively generates its own training data. Specifically, the generation of synthetic training images is controlled within an iterative training process. Valuable and/or informative object views are found in a low-dimensional rendering space and then added iteratively to the training set. In each iteration, new views are generated. A sparse training set is iteratively generated by searching for local minima of a classifier's output in a low-dimensional space of rendering parameters. An initial training set is generated. The classifier is trained using the training set. Local minima are found of the classifier's output in the low-dimensional rendering space. Images are rendered at the local minima. The newly-rendered images are added to the training set. The procedure is repeated so that the classifier is retrained using the modified training set.

Object recognition with 3D models

A face recognition system includes a component learning/extraction module, component classifier training module, knowledge base for component classification (KBCC), component extraction module (CEM), object identification training module (OITM), knowledge base for face identification (KBFI), and object identification module (OIM). The CEM receives image data of faces at various viewpoints and extracts outputs of classification of the component data, using the results of classifier training of the component data, stored in the KBCC. The OITM receives the outputs of classification of the component data and determines indicator component for each person by Bayesian estimation so that posterior probability of a predetermined attention class is maximized under the outputs of classification of the component data at various viewpoints. The KBFI stores indicator components for the individuals. The OIM receives the outputs of classification of the component data and identifies faces using the indicator components stored in the KBFI.

System and method for face recognition

Systems and methods are presented that determine components to use as examples to train a component-based face recognition system. In one embodiment, an initial component shape and size is determined, a training set is built, a component recognition classifier is trained, and the accuracy of the classifier is estimated. The component is then temporarily grown in each of four directions (up, down, left, and right) and the effect on the classifier's accuracy is determined. The component is then grown in the direction that maximizes the classifier's accuracy. The process can be performed multiple times in order to maximize the classifier's accuracy.

Systems and methods for training component-based object identification systems

Object recognition techniques are disclosed that provide both accuracy and speed. One embodiment of the present invention is an identification system. The system is capable of locating objects in images by searching for local features of an object. The system can operate in real-time. The system is trained from a set of images of an object or objects. The system computes interest points in the training images, and then extracts local image features (tokens) around these interest points. The set of tokens from the training images is then used to build a hierarchical model structure. During identification/detection, the system, computes interest points from incoming target images. The system matches tokens around these interest points with the tokens in the hierarchical model. Each successfully matched image token votes for an object hypothesis at a certain scale, location, and orientation in the target image. Object hypotheses that receive insufficient votes are rejected.

Growing community of inventors

Mountain View, CA, United States of America

Bernd Heisele