Honda Motor Co., Ltd.

Amazon Technologies, Inc.

Yan Lu

Kikuo Fujimura

Yi-Ting Chen

David Allen Fotland

Bernd Heisele

Ananth Ranganathan

Jiawei Huang

Aniket Murarka

Ishan Ankur Patil

Joydeep Biswas

Kun He

An autonomous mobile device moves through a physical space using simultaneous localization and mapping (SLAM) techniques. SLAM processes images from cameras to determine localization and trajectory of the device based on features that are assumed to be stationary. SLAM performance is improved by removing moving features from consideration. A first position of a feature at a first time and data from an inertial sensor are used to determine a predicted position at a second time. The predicted position is compared to a second position of the feature at the second time. This comparison takes into consideration an assumed Gaussian error distribution of how the positions are determined. If the predicted position differs from the second position by less than a threshold value, the feature may be determined to be stationary. The stationary features are then processed using SLAM to determine the localization and trajectory information.

System to determine stationary features by autonomous mobile device

Feature descriptor matching described herein may include receiving a first input image and a second input image. A feature detector may detect features from the first and second input images. A descriptor extractor may learn local feature descriptors from the features of the first and second input images based on a feature descriptor matching model trained using a ground truth data set. The descriptor extractor may determine a listwise mean average precision (mAP) rank of a pool of candidate image patches from the second input image with respect to a queried image patch from the first input image based on the feature descriptor matching model, the first set of local feature descriptors, and the second set of local feature descriptors. The descriptor matcher may generate a geometric transformation between the first input image and the second input image based on the listwise mAP and a convolutional neural network.

Feature descriptor matching

Systems and techniques for sign based localization are provided herein. Sign based localization may be achieved by capturing an image of an operating environment around a vehicle, extracting one or more text candidates from the image, detecting one or more line segments within the image, defining one or more quadrilateral candidates based on one or more of the text candidates, one or more of the line segments, and one or more intersections of respective line segments, determining one or more sign candidates for the image based on one or more of the quadrilateral candidates and one or more of the text candidates, matching one or more of the sign candidates against one or more reference images, and determining a location of the vehicle based on a match between one or more of the sign candidates and one or more of the reference images.

Sign based localization

The present disclosure relates to methods and systems for monocular localization in urban environments. The method may generate an image from a camera at a pose. The method may receive a pre-generated map, and determine features from the generated image based on edge detection. The method may predict a pose of the camera based on at least the pre-generated map, and determine features from the predicted camera pose. Further, the method may determine a Chamfer distance based upon the determined features from the image and the predicted camera pose, optimize the determined Chamfer distance based upon odometry information and epipolar geometry. Upon optimization, the method may determine an estimated camera pose.

Monocular localization in urban environments using road markings

Growing community of inventors

Newark, CA, United States of America

Yan Lu