Massachusetts Institute of Technology

Toyota Research Institute, Inc.

Antonio Torralba

Wojciech Matusik

Petr Kellnhofer

Simon A I Stent

Adrià Recasens

William T Freeman

Guy Rosman

Daniela Rus

Tomas Apostol Palacios

Gregory W Wornell

Sertac Karaman

Fredo Durand

Igor Gilitschenski

Alexander Amini

Simon Stent

Wan Shou

Felix Maximilian Naser

Michael J Foshey

Yiyue Luo

Pratyusha Sharma

Yunzhu Li

Systems and methods are provided for estimating 3D poses of a subject based on tactile interactions with the ground. Test subject interactions with the ground are recorded using a sensor system along with reference information (e.g., synchronized video information) for use in correlating tactile information with specific 3D poses, e.g., by training a neural network based on the reference information. Then, tactile information received in response to a given subject interacting with the ground can be used to estimate the 3D pose of the given subject directly, i.e., without reference to corresponding reference information. Certain exemplary embodiments use a sensor system in the form of a pressure sensing carpet or mat, although other types of sensor systems using pressure or other sensors can be used in various alternative embodiments.

Systems and methods for estimating 3D position and movement from tactile signals

The present disclosure provides systems and methods to detect occluded objects using shadow information to anticipate moving obstacles that are occluded behind a corner or other obstacle. The system may perform a dynamic threshold analysis on enhanced images allowing the detection of even weakly visible shadows. The system may classify an image sequence as either 'dynamic' or 'static', enabling an autonomous vehicle, or other moving platform, to react and respond to a moving, yet occluded object by slowing down or stopping.

Systems and methods of detecting moving obstacles

A method includes receiving, with a computing device, an image, identifying one or more salient features in the image, and generating a saliency map of the image including the one or more salient features. The method further includes sampling the image based on the saliency map such that the one or more salient features are sampled at a first density of sampling and at least one portion of the image other than the one or more salient features are sampled at a second density of sampling, where the first density of sampling is greater than the second density of sampling, and storing the sampled image in a non-transitory computer readable memory.

Systems and methods for saliency-based sampling layer for neural networks

A system includes a camera positioned in an environment to capture image data of a subject; a computing device communicatively coupled to the camera, the computing device comprising a processor and a non-transitory computer-readable memory; and a machine-readable instruction set stored in the non-transitory computer-readable memory. The machine-readable instruction set causes the computing device to perform at least the following when executed by the processor: receive the image data from the camera; analyze the image data captured by the camera using a neural network trained on training data generated from a 360-degree panoramic camera configured to collect image data of a subject and a visual target that is moved about an environment; and predict a gaze direction vector of the subject with the neural network.

Opengaze: gaze-tracking in the wild

A system for determining the gaze direction of a subject includes a camera, a computing device and a machine-readable instruction set. The camera is positioned in an environment to capture image data of head of a subject. The computing device is communicatively coupled to the camera and the computing device includes a processor and a non-transitory computer-readable memory. The machine-readable instruction set is stored in the non-transitory computer-readable memory and causes the computing device to: receive image data from the camera, analyze the image data using a convolutional neural network trained on an image dataset comprising images of a head of a subject captured from viewpoints distributed around up to 360-degrees of head yaw, and predict a gaze direction vector of the subject based upon a combination of head appearance and eye appearance image data from the image dataset.

Systems and methods for gaze tracking from arbitrary viewpoints

Growing community of inventors

Cambridge, MA, United States of America

Antonio Torralba