Massachusetts Institute of Technology

Toyota Research Institute, Inc.

Eth Zurich

Disney Enterprises, Inc.

Igor Gilitschenski

Daniela Rus

Sertac Karaman

Guy Rosman

Alexander Amini

John J Leonard

Felix Maximilian Naser

William T Freeman

Stephen G McGill

Luke S Fletcher

Paul A Beardsley

Gregory W Wornell

Fredo Durand

Roland Siegwart

Alyssa Pierson

Xin Huang

Antonio Torralba

Moses Theodore Ort

Stephen G Mcgill

Minoru Brandon Araki

Anurag Sai Vempati

Brian C Williams

Cesar Cadena

Jacob Phillips

Juan Nieto

Arjun Gupta

Brandon Araki

Xiao Li

Cristian-Ioan Vasile

Christina Liao

Ashkan Mohammadzadeh Jasour

Andrei Cramariuc

Julia Moseyko

A controller for an autonomous vehicle is trained using simulated paths on a roadway and simulated observations that are formed by transforming images previously acquired on similar paths on that roadway. Essentially an unlimited number of paths may be simulated, enabling optimization approaches including reinforcement learning to be applied to optimize the controller.

Simulation-based training of an autonomous vehicle

Systems and methods for predicting a trajectory of a moving object are disclosed herein. One embodiment downloads, to a robot, a probabilistic hybrid discrete-continuous automaton (PHA) model learned as a deep neural network; uses the deep neural network to infer a sequence of high-level discrete modes and a set of associated low-level samples, wherein the high-level discrete modes correspond to candidate maneuvers for the moving object and the low-level samples are candidate trajectories; uses the sequence of high-level discrete modes and the set of associated low-level samples, via a learned proposal distribution in the deep neural network, to adaptively sample the sequence of high-level discrete modes to produce a reduced set of low-level samples; applies a sample selection technique to the reduced set of low-level samples to select a predicted trajectory for the moving object; and controls operation of the robot based, at least in part, on the predicted trajectory.

Systems and methods for predicting the trajectory of a moving object

A method of generating an output trajectory of an ego vehicle is described. The method includes extracting high-level features from a bird-view image of a traffic environment of the ego vehicle. The method also includes generating, using an automaton generative network, an automaton including an automaton state distribution describing a behavior of the ego vehicle in the traffic environment according to the high-level features. The method further includes generating the output trajectory of the ego vehicle according to extracted bird-view features of the bird-view image and the automaton state distribution describing the behavior of the ego vehicle in the traffic environment.

Method for learning an explainable trajectory generator using an automaton generative network

Systems and methods described herein relate to estimating risk associated with a vehicular maneuver. One embodiment acquires a geometric representation of an intersection including a lane in which a vehicle is traveling and at least one other lane; discretizes the at least one other lane into a plurality of segments; determines a trajectory along which the vehicle will travel; estimates a probability density function for whether a road agent external to the vehicle is present in the respective segments; estimates a traffic-conflict probability of a traffic conflict in the respective segments conditioned on whether an external road agent is present; estimates a risk associated with the vehicle following the trajectory by integrating a product of the probability density function and the traffic-conflict probability over the at least one other lane and the plurality of segments; and controls operation of the vehicle based, at least in part, on the estimated risk.

Systems and methods for estimating the risk associated with a vehicular maneuver

The present disclosure provides systems and methods to detect occluded objects using shadow information to anticipate moving obstacles that are occluded behind a corner or other obstacle. The system may perform a dynamic threshold analysis on enhanced images allowing the detection of even weakly visible shadows. The system may classify an image sequence as either 'dynamic' or 'static', enabling an autonomous vehicle, or other moving platform, to react and respond to a moving, yet occluded object by slowing down or stopping.

Systems and methods of detecting moving obstacles

Systems and methods for predicting the trajectory of an object are disclosed herein. One embodiment receives sensor data that includes a location of the object in an environment of the object; accesses a location-specific latent map, the location-specific latent map having been learned together with a neural-network-based trajectory predictor during a training phase, wherein the neural-network-based trajectory predictor is deployed in a robot; inputs, to the neural-network-based trajectory predictor, the location of the object and the location-specific latent map, the location-specific latent map providing, to the neural-network-based trajectory predictor, a set of location-specific biases regarding the environment of the object; and outputs, from the neural-network-based trajectory predictor, a predicted trajectory of the object.

Systems and methods for predicting the trajectory of an object with the aid of a location-specific latent map

An approach to place recognition from an image makes use of the detection of objects at a set of known places as well as at an unknown place. Images of the detected objects in an image of the unknown place are processed to yield respective numerical descriptors, and these descriptors are used to compare the unknown place to the known places to recognize the unknown place. At least some embodiments make use of a trained parameterized image processor to transform an image of an object to an object descriptor, and the training of the processor is meant to preserve distinctions between different instances of a type of object, as well as distinctions between entirely different types of objects.

Visual object instance descriptor for place recognition

A method of non-line-of-sight (NLoS) obstacle detection for an ego vehicle is described. The method includes capturing a sequence of images over a period with an image capture device. The method also includes storing the sequence of images in a cyclic buffer. The method further includes registering each image in the cyclic buffer to a projected image. The method includes performing the registering by estimating a homography H for each frame of the sequence of images to project to a view point of a first frame in the sequence of images and remove motion of the ego vehicle in the projected image. The method also includes enhancing the projected image. The method further includes classifying the projected image based on a scene determination. The method also includes issuing a control signal to the vehicle upon classifying the projected image.

Infrastructure-free NLoS obstacle detection for autonomous cars

According to one implementation, a robotic system for performing large-scale environmental mapping in real-time includes a mobile reconnaissance unit having a color sensor, a depth sensor, and a graphics processing unit (GPU) with a GPU memory, and a navigation unit communicatively coupled to the mobile reconnaissance unit and having a central processing unit (CPU) with a CPU memory. The robotic system begins a three-dimensional (3D) scan of an environment of the mobile reconnaissance unit using the color sensor and the depth sensor, and generates mapping data for populating a volumetric representation of the environment. The robotic system continues the 3D scan of the environment using the color sensor and the depth sensor, updates the mapping data, and partitions the volumetric representation between the GPU memory and the CPU memory based on a memory allocation criteria.

Large-scale environmental mapping in real-time by a robotic system

Growing community of inventors

Cambridge, MA, United States of America

Igor Gilitschenski