Mitsubishi Electric Research Laboratories, Inc.

Mitsubishi Electric Corporation

Yuichi Taguchi

Srikumar Ramalingam

Oncel C Tuzel

Chen Feng

Amit Kumar Agrawal

Ming-Yu Liu

Esra Cansizoglu

Ashok Veeraraghavan

Tim Marks

Vineet R Kamat

Cuneyt Oncel Tuzel

Sergio Salvatore Caccamo

Nitesh Shroff

Haruyuki Iwama

Yong Xiao

Ramesh Raskar

Anthony Vetro

Raquel Urtasun

Petros T Boufounos

Hassan Mansour

Dehong Liu

Jay E Thornton

John R Hershey

Arvind Raghunathan

Christopher Laughman

Ulugbek Kamilov

Yohei Miki

Hiroyuki Tsutada

Ryosuke Kawanishi

Yanting Ma

Yukiyasu Domae

Dan J Burns

Tyler W Garaas

Teng-Yok Lee

Keisuke Sambongi

Wim Abbeloos

Yu Enomoto

Jaishanker K Pillai

Kilho Son

Michel Antunes

Esra Ataer-Cansizoglu

Jian Yao

Yong-Dian Jian

Changhyun Choi

Menglong Zhu

Khalid Yousif

Sonali Patil

Bharath Sankaran

Maja Krivokuca

Robert Cohen

A method for machining a workpiece by a numerical control (NC) machine constructs a model of a scene from one or multiple images including at least a part of the workpiece and a part of the NC machine and registers a model of the NC machine with the model of the scene to produce a first transformation between a coordinate system of the model of the scene and a coordinate system of the NC machine. The method also detects the workpiece in the model of the scene to produce a second transformation between the coordinate system of the model of the scene and a coordinate system of the workpiece and combines the first and the second transformations to register the coordinate system of the workpiece with the coordinate system of the NC machine. The registered workpiece is machined with a tool of the NC machine.

System and method for performing operations of numerical control machines

A method calibrates one or more cameras, wherein the one or more cameras acquire images of a scene for different viewpoints, and wherein the images are non-overlapping, by first constructing a 3D model of the scene using a calibration camera that is independent of the one or more cameras, and a simultaneous localization and mapping (SLAM) procedure. 2D-to-3D correspondences between each of the images and the 3D model are determined. Then, calibration parameters of each of the one or more cameras are estimated using a 2D-to-3D registration procedure.

Method for calibrating cameras with non-overlapping views

Provided is an elevator shaft dimensions measurement device including: a plurality of 3-D distance image sensors which are arranged on a circumference of the same circle, facing the direction of the center of the circle and inclined at an elevation angle with respect to a horizontal plane, and which output measurement data by capturing an image of a pattern irradiated onto the inner walls of an elevator shaft that are imaging objects; and a computer which integrates the measurement data output from the plurality of 3-D distance image sensors at a plurality of height positions in the elevator shaft, generates first integrated measurement data covering 360 degrees in the horizontal direction, aligns the first integrated measurement data to create second integrated measurement data after the alignment, and calculates the dimensions of the elevator shaft on the basis of the second integrated measurement data after the alignment.

Elevator shaft dimensions measurement device and elevator shaft dimensions measurement method

A method and system detects and localizes multiple instances of an object by first acquiring a frame of a three-dimensional (3D) scene with a sensor, and extracting features from the frame. The features are matched according to appearance similarity and triplets are formed among matching features. Based on 3D locations of the corresponding points in the matching triplets, a geometric transformation is computed. Matching triplets are clustered according to the computed geometric transformations. Since the set of features coining from two different object instances should have a single geometric transform, the output of clustering provides the features and poses of each object instance in the image.

Imaging system and method for object detection and localization

A first T-spline fitting system includes a three-dimensional (3D) sensor, a processor and a memory storing a program for representing a point cloud of the scene using T-spline representation. The system executes acquiring an image of the scene using the 3D sensor, generating the point cloud from the image, determining a parameterized domain of the point cloud by mapping data points of the point cloud to values of parameters of the parameterized domain, partitioning recursively the parameterized domain into a set of patches, connecting the patches to form a T-mesh defining topology of control points in the parameterized domain, and fitting a T-spline surface into the point cloud according to the T-mesh to determine positions of the control points.

Fast T-spline fitting system and method

An imaging system for localization and mapping of a scene including static and dynamic objects. A sensor acquires a sequence of frames in motion or stationary. A memory to store a static map of static objects and an object map of each dynamic object in the scene. The static map includes a set of landmarks, and the object map includes a set of landmarks and a set of segments. A localizer registers keypoints of the frame with landmarks in the static map using frame-based registration and to register some segments in the frame with segments in the object map using a segment-based registration. A mapper to update each object map with keypoints forming each segment and keypoints registered with the corresponding object map according to the segment-based registration, and to update the static map with the remaining keypoints in the frame using the keypoints registered with the static map.

Method and system for concurrent reconstruction of dynamic and static objects

A system for reconstructing a three-dimensional (3D) model of a scene including a point cloud having points identified by 3D coordinates includes at least one sensor to acquire a set of images of the scene from different poses defining viewpoints of the images and a memory to store the set of images and the 3D model of the scene. The system also includes a processor operatively connected to the memory and coupled with stored instructions to transform the images from the set of images to produce a set of virtual images of the scene viewed from virtual viewpoints; compare at least some features from the images and the virtual images to determine the viewpoint of each image in the set of images; and update 3D coordinates of at least one point in the model of the scene to match coordinates of intersections of ray back-projections from pixels of at least two images corresponding to the point according to the viewpoints of the two images.

System and method for virtually-augmented visual simultaneous localization and mapping

Systems and methods for an imaging system having a memory with a historical localization dictionary database having geo-located driving image sequences, such that each reference image is applied to a threshold to produce a binary representation. A sensor to acquire a sequence of input images of a dynamic scene. An encoder to determine, for each input image in the sequence, a histogram of each input image indicating a number of vertical edges at each bin of the input image and to threshold the histogram to produce a binary representation of the input image. A visual odometer to compare the binary representations of each input image and each reference image, by matching an input image against a reference image. Wherein the visual odometer determines a location of the input image based on a match between the input image and the reference image.

Barcode: global binary patterns for fast visual inference

A system controls motion of vehicle according to a first parking path ending at a target state and determines a second parking path from a current state to the target state using a model of the parking space. The system acquires images of the parking space during the motion of the vehicle along the first parking path and constructs the model of the parking space. The model is used to generate a set of virtual images of the environment of the parking space as viewed from virtual viewpoints outside of the first parking path. The current state of the vehicle is determined by comparing a current image of the parking space with at least one virtual image. The second parking path is determined from the current state to the target state.

Vehicle automated parking system and method

Systems and methods for image processing including a memory having stored multi-angled view images of a scene. Each multi-angled view image includes pixels, and at least one multi-angled view image includes a clouded area in the scene, resulting in missing pixels. A processor configured to align three multi-angled view images in the multi-angled view images to a target view angle of the scene, to form a set of aligned multi-angled view images representing a target point of view of the scene, at least one aligned multi-angled view image of the at least three multi-angled view images, has missing pixels due to the clouded area. Form a matrix using vectorized aligned multi-angled view images, wherein the matrix is incomplete due to the missing pixels. Complete the matrix using a matrix completion to combine the aligned multi-angled view images to produce a fused image of the scene without the clouded area.

Systems and methods of fusing multi-angle view HD images based on epipolar geometry and matrix completion

Systems for completing missing points in a three-dimensional (3D) point cloud. The systems include a 3D sensor for measuring point data of objects in a scene, a processor in connection with the 3D sensor and a memory storing a program and the point data, and a display monitor connected to the processor and the 3D sensor for displaying a completed 3D point cloud, wherein the processor executes instruction steps of the program. The instruction steps include acquiring the point data of the objects to generate the 3D point cloud, wherein the point data include a set of points on the objects, extracting planar segments from the 3D point cloud, identifying connectivity relations and missing points among the planar segments, and filling the missing points in the 3D point cloud to generate the completed 3D point cloud using the planar segments and the connectivity relations.

Method and system for completing point clouds using planar segments

An image registrations includes determining a first binary descriptor of a first key point in a first image, determining a second binary descriptor of a second key point in a second image, determining a weighted Hamming distance between the first binary descriptor and the second binary descriptor, and registering the first key point with the second key point when the weighted Hamming distance is below a noise threshold. At least one element in the first or the second binary descriptor is a result of a comparison of a difference between intensities of at least two pixels of the first or the second image with a threshold. At least two weights of the weighted Hamming distance for comparing at least two elements of the first or the second binary descriptors are different.

Method and system for image registrations

A method upsamples an image using a non-linear fully connected neural network to produce only global details of an upsampled image and interpolates the image to produce a smooth upsampled image. The method concatenates the global details and the smooth upsampled image into a tensor and applies a sequence of nonlinear convolutions to the tensor using a convolutional neural network to produce the upsampled image.

Image upsampling using global and local constraints

A method for denoising a range image acquired by a time-of-flight (ToF) camera by first determining locations of edges, and a confidence value of each pixel, and based on the locations of the edges, determining geodesic distances of neighboring pixels. Based on the confidence values, reliabilities of the neighboring pixels are determined and scene dependent noise is reduced using a filter.

Method for denoising time-of-flight range images

A method and system for registering data by first acquiring the data from a scene by a sensor at different viewpoints, and extracting, from the data, three-dimensional (3D) points and 3D lines, and descriptors associated with the 3D points and the 3D lines. A first set of primitives represented in a first coordinate system of the sensor is selected, wherein the first set of primitives includes at least three 3D points. A second set of primitives represented in a second coordinate system is selected, wherein the second set of primitives includes any combination of 3D points and 3D lines to obtain at least three primitives. Then, using the first set of primitives and the second set of primitives, the 3D points are registered with each other, and the 3D points with the 3D lines to obtain registered primitives, wherein the registered primitives are used in a simultaneous localization and mapping (SLAM) system.

System and method for hybrid simultaneous localization and mapping of 2D and 3D data acquired by sensors from a 3D scene

Growing community of inventors

Arlington, MA, United States of America

Yuichi Taguchi