Mitsubishi Electric Research Laboratories, Inc.

Other

Oncel C Tuzel

Fatih M Porikli

Ming-Yu Liu

Yuichi Taguchi

Tim Marks

Amit Kumar Agrawal

Srikumar Ramalingam

Michael J Jones

Jay E Thornton

Ashok Veeraraghavan

Aswin C Sankaranarayanan

Raviteja Vemulapalli

Jie Ni

Zafer Sahinoglu

Arvind Raghunathan

Jeroen Van Baar

Dirk Brinkman

Bharat Singh

Amir Massoud Farahmand

Jaishanker K Pillai

Changhyun Choi

Chinmay Hegde

Weiguang Si

Kota Hara

Salil Tambe

Abhishek Sharma

A method and system detects actions of an object in a scene by first acquiring a video of the scene as a sequence of images, wherein each image includes pixels, wherein the video is partitioned into chunks. The object in the video is tracked. For each object and each chunk of the video, trajectories of the pixels within a bounding box located over the object are tracked, and cropped trajectories and cropped images for one or more images in the chunk are produced using the bounding box. Then, the cropped trajectories and cropped images are passed to a recurrent neural network (RNN) that outputs a relative score for each action of interest.

Method and system for detecting actions in videos

A method detects an object in an image. The method extracts a first feature vector from a first region of an image using a first subnetwork and determines a second region of the image by processing the first feature vector with a second subnetwork. The method also extracts a second feature vector from the second region of the image using the first subnetwork and detects the object using a third subnetwork on a basis of the first feature vector and the second feature vector to produce a bounding region surrounding the object and a class of the object. The first subnetwork, the second subnetwork, and the third subnetwork form a neural network. Also, a size of the first region differs from a size of the second region.

Object detection system and object detection method

A computer-implemented method generates a multimodal digital image by processing a vector with a first neural network to produce a first modality of the digital image and processing the vector with a second neural network to produce a second modality of the digital image. A structure and a number of layers of the first neural network are identical to a structure and a number of layers of the second neural network. Also, at least one layer in the first neural network has parameters identical to parameters of a corresponding layer in the second neural network, and at least one layer in the first neural network has parameters different from parameters of a corresponding layer in the second neural network.

Method and system for generating multimodal digital images

A method upsamples an image using a non-linear fully connected neural network to produce only global details of an upsampled image and interpolates the image to produce a smooth upsampled image. The method concatenates the global details and the smooth upsampled image into a tensor and applies a sequence of nonlinear convolutions to the tensor using a convolutional neural network to produce the upsampled image.

Image upsampling using global and local constraints

A method labels an image of a street view by first extracting, for each pixel, an appearance feature for inferring a semantic label, a depth feature for inferring a depth label. Then, a column-wise labeling procedure is applied to the features to jointly determine the semantic label and the depth label for each pixel using the appearance feature and the depth feature, wherein the column-wise labeling procedure is according to a model of the street view, and wherein each column of pixels in the images includes at most four layers.

Method for labeling images of street scenes

A computer-implemented method for semantic segmentation of an image determines unary energy of each pixel in an image using a first subnetwork, determines pairwise energy of at least some pairs of pixels of the image using a second subnetwork, and determines, using a third subnetwork, an inference on a Gaussian random field (GRF) minimizing an energy function including a combination of the unary energy and the pairwise energy. The GRF inference defining probabilities of semantic labels for each pixel in the image, and the method converts the image into a semantically segmented image by assigning to a pixel in the semantically segmented image a semantic label having the highest probability for a corresponding pixel in the image among the probabilities determined by the third subnetwork. The first subnetwork, the second subnetwork, and the third subnetwork are parts of a neural network.

System and method for semantic segmentation using Gaussian random field network

A sensor acquires an input image X of a scene. The image includes noise with a variance &#x3c3;. A deep Gaussian conditional random field (GCRF) network is applied to the input image to produce an output image Y, where the output image is denoised, and wherein the deep GCRF includes a prior generation (PgNet) network followed by an inference network (InfNet), wherein the PgNet produces patch covariance priors &#x3a3;for patches centered on every pixel (i,j) in the input image, and wherein the InfNet is applied to the patch covariance priors and the input image to solve the GCRF.

Method and system for denoising images using deep Gaussian conditional random field network

A method for face alignment operates on a face image and a set of intitial landmark locations by first aligning globally the initial locations to a set of landmark locations of a face with a prototype shape to obtain global alignment parameters, and then warping the initial locations and the image from a coordinate frame of the image to a coordinate frame of the prototype shape according to the global alignment parameters to obtain warped landmark locations and a warped face image. Features are extracted from the warped face image at the warped landmark locations, and a regression function is applied to the features to obtain updated landmark locations in the coordinate frame of the prototype shape. Finally, the updated landmark locations in the coordinate frame of the prototype shape are warped to the coordinate frame of the image, to obtain updated landmark locations.

Method for estimating locations of facial landmarks in an image of a face using globally aligned regression

A vehicle includes a set of devises for determining a state of the vehicle at an instant of time and a processor operatively connected to a memory for processing parameters of the state of the vehicle collected over a plurality of instances of time for a period of time from an initial instance of time till a current instant of time to determine a set of driving conditions leading to a current state of the vehicle at the current instant of time. Each driving condition indicates a condition of an operation of the vehicle for the period of time. The vehicle also includes a communication module for outputting the current state of the vehicle determined by the set of devices at the current instant of time and for outputting the set of driving conditions leading to the current state of the vehicle.

System and method for testing and evaluating vehicle components

A resolution of a low resolution depth image is increased by applying joint geodesic upsampling to a high resolution image to obtain a geodesic distance map. Depths in the low resolution depth image are interpolated using the geodesic distance map to obtain a high resolution depth image. The high resolution image can be a gray scale or color image, or a binary boundary map. The low resolution depth image can be acquired by any type of depth sensor.

Method for increasing resolutions of depth images

A method semantically labels an image acquired of a scene by first obtaining a local semantic feature for each local region in the image. The local semantic features are combined recursively to form intermediate segments until a semantic feature for the entire image is obtained. Then, the semantic feature for the entire image is decombined recursively into intermediate segments until an enhanced semantic feature for each local region is obtained. Then, each local region is labeled according to the enhanced semantic feature.

Method for semantically labeling an image of a scene using recursive context propagation

Three-dimensional (3D) geometric boundaries are detected in images of a scene that undergoes varying lighting conditions caused by light sources in different positions, from a set of input images of the scene illuminated by at least two different lighting conditions. The images are aligned, e.g., acquired by a stationary camera, so that pixels at the same location in all of the input images correspond to the same point in the scene. For each location, a patch of corresponding pixels centered at the location is extracted from each input image. For each location, a confidence value that there is a 3D geometric boundary at the location is determined.

Method for detecting 3D geometric boundaries in images of scenes subject to varying lighting

A set of nonnegative lighting basis images representing a scene illuminated by a set of stationary light sources is recovered from a set of input images of the scene that were acquired by a stationary camera. Each image is illuminated by a combination of the light sources, and at least two images in the set are illuminated by different combinations. The set of input images is factorized into the nonnegative lighting basis images and a set of indicator coefficients, wherein each lighting basis image corresponds to an appearance of the scene illuminated by one of the light sources, and wherein each indicator coefficient indicates a contribution of one of the light sources to one of the input images.

Method for factorizing images of a scene into basis images

A method for determining a pose of an object in a scene by determining a set of scene features from data acquired of the scene and matching the scene features to model features to generate weighted candidate poses when the scene feature matches one of the model features, wherein the weight of the candidate pose is proportional to the model weight. Then, the pose of the object is determined from the candidate poses based on the weights.

Method for determining object poses using weighted features

A method denoises a noisy image by, for each pixel in the noisy image, first constructing a key from a patch, wherein the patch includes locally neighboring pixels around the pixel. A function is selected from a function library using the key. Then, the function is applied to the patch to generate a corresponding noise free pixel for the pixel.

Growing community of inventors

Piscataway, NJ, United States of America

Oncel C Tuzel