Mitsubishi Electric Research Laboratories, Inc.

Mitsubishi Electric Corporation

Srikumar Ramalingam

Yuichi Taguchi

Amit Kumar Agrawal

Matthew E Brand

Ming-Yu Liu

Oncel C Tuzel

Chen Feng

Esra Cansizoglu

Ashok Veeraraghavan

Cuneyt Oncel Tuzel

Ramesh Raskar

Raquel Urtasun

Daniel Nikolaev Nikovski

Jay E Thornton

Arvind Raghunathan

Christopher Laughman

Yohei Miki

Dan J Burns

Teng-Yok Lee

Tyler W Garaas

Jaishanker K Pillai

Haruyuki Iwama

Nitesh Shroff

Sofien Bouaziz

Wenzhen Yuan

Bharath Sankaran

Gim Hee Lee

Michel Antunes

Esra Ataer-Cansizoglu

Jian Yao

Khalid Yousif

Yong-Dian Jian

Changhyun Choi

Sonali Patil

Menglong Zhu

Zhiding Yu

A method calibrates one or more cameras, wherein the one or more cameras acquire images of a scene for different viewpoints, and wherein the images are non-overlapping, by first constructing a 3D model of the scene using a calibration camera that is independent of the one or more cameras, and a simultaneous localization and mapping (SLAM) procedure. 2D-to-3D correspondences between each of the images and the 3D model are determined. Then, calibration parameters of each of the one or more cameras are estimated using a 2D-to-3D registration procedure.

Method for calibrating cameras with non-overlapping views

A system for reconstructing a three-dimensional (3D) model of a scene including a point cloud having points identified by 3D coordinates includes at least one sensor to acquire a set of images of the scene from different poses defining viewpoints of the images and a memory to store the set of images and the 3D model of the scene. The system also includes a processor operatively connected to the memory and coupled with stored instructions to transform the images from the set of images to produce a set of virtual images of the scene viewed from virtual viewpoints; compare at least some features from the images and the virtual images to determine the viewpoint of each image in the set of images; and update 3D coordinates of at least one point in the model of the scene to match coordinates of intersections of ray back-projections from pixels of at least two images corresponding to the point according to the viewpoints of the two images.

System and method for virtually-augmented visual simultaneous localization and mapping

Systems and methods for an imaging system having a memory with a historical localization dictionary database having geo-located driving image sequences, such that each reference image is applied to a threshold to produce a binary representation. A sensor to acquire a sequence of input images of a dynamic scene. An encoder to determine, for each input image in the sequence, a histogram of each input image indicating a number of vertical edges at each bin of the input image and to threshold the histogram to produce a binary representation of the input image. A visual odometer to compare the binary representations of each input image and each reference image, by matching an input image against a reference image. Wherein the visual odometer determines a location of the input image based on a match between the input image and the reference image.

Barcode: global binary patterns for fast visual inference

A image processing system for multi-label semantic edge detection in an image includes an image interface to receive an image of a scene including at least one object, a memory to store a neural network trained for performing a multi-label edge classification of input images assigning each pixel of edges of objects in the input images into one or multiple semantic classes, a processor to transform the image into a multi-label edge-map using the neural network detecting an edge of the object in the image and assigning multiple semantic labels to at least some pixels forming the edge, and an output interface to render the multi-label edge-map.

Multi-label semantic boundary detection system

A method and system estimates a three-dimensional (3D) pose of a sensor by first acquiring scene data of a 3D scene by the sensor. Two-dimensional (2D) lines are detected in the scene data and the 2D lines are matched to 3D lines of a 3D model of the scene to produce matching lines. Then, the 3D pose of the sensor is estimated using the matching lines.

Pose estimation using sensors

Systems and Methods for controlling a movement of cars of an elevator system. A processor determines for each car an individual waiting time of each hall call. Determines for each pair of hall calls assigned for each car, a pairwise delay over the individual waiting time of each hall call in the pair caused by a joint assignment of the car to the pair of the hall calls. Approximate a cumulative waiting time of an assignment of the cars to the hall calls as a sum of individual waiting times for each hall call with the assigned car and a sum of all pairwise delays determined between all pairs of hall calls assigned to the same car. Determine the assignment of the cars using a greedy optimization algorithm that greedily assigns hall calls to the cars to minimize the approximated cumulative waiting time, and control the movement of the cars.

System and method for group elevator scheduling based on submodular optimization

An image registrations includes determining a first binary descriptor of a first key point in a first image, determining a second binary descriptor of a second key point in a second image, determining a weighted Hamming distance between the first binary descriptor and the second binary descriptor, and registering the first key point with the second key point when the weighted Hamming distance is below a noise threshold. At least one element in the first or the second binary descriptor is a result of a comparison of a difference between intensities of at least two pixels of the first or the second image with a threshold. At least two weights of the weighted Hamming distance for comparing at least two elements of the first or the second binary descriptors are different.

Method and system for image registrations

A method reconstructs a scene as a three-dimensional (3D) model by first acquiring a sequence of images of the scene with a camera. Then, feature point matches or line matches are extracted from the sequence of images, variables for camera optical centers and 3D points are initialized using random values, and n copies of the variables are made depending on an availability of n constraints. The n copies are projected to satisfy each of the n constraints. Then, the n copies are replaced with averages of the copies and the projecting and the replacing are repeated until convergence to provide the 3D model.

Method and system for reconstructing scenes as 3D models from sequences of images using constraint satisfaction

A method labels an image of a street view by first extracting, for each pixel, an appearance feature for inferring a semantic label, a depth feature for inferring a depth label. Then, a column-wise labeling procedure is applied to the features to jointly determine the semantic label and the depth label for each pixel using the appearance feature and the depth feature, wherein the column-wise labeling procedure is according to a model of the street view, and wherein each column of pixels in the images includes at most four layers.

Method for labeling images of street scenes

A method and system for registering data by first acquiring the data from a scene by a sensor at different viewpoints, and extracting, from the data, three-dimensional (3D) points and 3D lines, and descriptors associated with the 3D points and the 3D lines. A first set of primitives represented in a first coordinate system of the sensor is selected, wherein the first set of primitives includes at least three 3D points. A second set of primitives represented in a second coordinate system is selected, wherein the second set of primitives includes any combination of 3D points and 3D lines to obtain at least three primitives. Then, using the first set of primitives and the second set of primitives, the 3D points are registered with each other, and the 3D points with the 3D lines to obtain registered primitives, wherein the registered primitives are used in a simultaneous localization and mapping (SLAM) system.

System and method for hybrid simultaneous localization and mapping of 2D and 3D data acquired by sensors from a 3D scene

A method reconstructs at three-dimensional (3D) real-world scene from a single two-dimensional (2D) image by identifying junctions satisfying geometric constraint of the scene based on intersecting lines, vanishing points, and vanishing lines that are orthogonal to each other. Possible layouts of the scene are generated by sampling the 2D image according to the junctions. Then, an energy function is maximized to select an optimal layout from the possible layouts. The energy function use's a conditional random field (CRF) model to evaluate the possible layouts.

Method for reconstructing 3D scenes from 2D images

A method determines motion between first and second coordinate systems by first extracting first and second sets of keypoints from first and second images acquired of a scene by a camera arranged on a moving object. First and second poses are determined from the first and second sets of keypoints. A score for each possible motion between the first and the second poses is determined using a scoring function and a pose-transition graph constructed from training data where each node in the post-transition graph represents a relative pose and each edge represents a motion between two consecutive relative poses. Then, based on the score, a best motion is selected as the motion between the first and second coordinate systems.

On-road stereo visual odometry without explicit pose determinations

A method reconstructs a three-dimensional (3D) scene as a three-dimensional (3D) model by first acquiring red, green, blue, and depth (RGBD) images with a sensor, and wide field-of-view (FOV) red, green, and blue (RGB) images with a camera from different viewpoints. Pair-wise point matches among the RGBD images and the RGB images are identified, and a graph of nodes and edges is constructed using the pair-wise point matches. Initial poses of the RGBD images and the RGB images are estimated using pose constraints associated with the edges, and an initial 3D model of the scene is estimated using the estimated poses. Then, erroneous edges are removed in the graph, and bundle adjustment is applied to the initial poses and the initial 3D model to obtain refined poses and the 3D model of the scene.

Method for reconstructing a 3D scene as a 3D model using images acquired by 3D sensors and omnidirectional cameras

A method determines a motion between a first and second coordinate system, by first extracting a first set of primitives from a 3D image acquired in the first coordinate system from an environment, and extracting a second set of primitives from a 3D image acquired in the second coordinate system from the environment. Motion hypotheses are generated for different combinations of the first and second sets of primitives using a RANdom SAmple Consensus procedure. Each motion hypothesis is scored using a scoring function learned using parameter learning techniques. Then, a best motion hypothesis is selected as the motion between the first and second coordinate system.

Method for stereo visual odometry using points, lines and planes

A method registers data using a set of primitives including points and planes. First, the method selects a first set of primitives from the data in a first coordinate system, wherein the first set of primitives includes at least three primitives and at least one plane. A transformation is predicted from the first coordinate system to a second coordinate system. The first set of primitives is transformed to the second coordinate system using the transformation. A second set of primitives is determined according to the first set of primitives transformed to the second coordinate system. Then, the second coordinate system is registered with the first coordinate system using the first set of primitives in the first coordinate system and the second set of primitives in the second coordinate system. The registration can he used to track a pose of a camera acquiring the data.

Growing community of inventors

Cambridge, MA, United States of America

Srikumar Ramalingam