Toyota Research Institute, Inc.

Massachusetts Institute of Technology

Luke S Fletcher

Guy Rosman

Stephen G McGill

John J Leonard

Simon A I Stent

Xin Huang

Gill A Pratt

Kazunori Nimura

Yusuke Sawamura

Stephen G Mcgill

Simon Stent

Deepak Edakkattil Gopinath

Daniela Rus

Nikos Arechiga Gonzalez

Jonathan Decastro

Sertac Karaman

Igor Gilitschenski

Alyssa Pierson

Moses Theodore Ort

John H Gideon

Brian C Williams

Jonathan A DeCastro

Minoru Brandon Araki

Daniel Stonier

Brent Schlotfeldt

Katsuya Terahata

A method for indicating occlusion information at an ego agent includes observing a spatial area from a first viewpoint of one or more first sensors associated with the ego agent. The method also includes identifying the spatial area as an occluded area in accordance with observing the spatial area from a second viewpoint of the one or more first sensors after observing the spatial area from the first viewpoint. The method further includes receiving, from a target agent, a message indicating the spatial area is occluded from one or more second sensors associated with the target agent. The method still further includes transmitting, to the target agent in accordance with receiving the message, the occlusion information indicating information associated the spatial area based on identifying the spatial area as the occluded area.

Peer-to-peer occupancy estimation

Systems and methods for generating efficient planning routes for vehicles, including autonomous and semi-autonomous vehicles are presented. A route planner may generate dynamic maps and routes that reduces the uncertainty of road-agent environmental and behavioral data in an efficient manner. Route planning may be accomplished using a statistical approach in which known data from one geographic or behavioral feature set may be used and relied upon by a vehicle in another geographical and behavioral context to estimate the environmental and behavioral data relevant to the vehicles current operation.

Dynamic map generation with focus on construction and localization field of technology

A system includes a camera configured to capture image data of an environment, a monitoring system configured to generate a gaze sequences of a subject, and a computing device communicatively coupled to the camera and the monitoring system. The computing device is configured to receive the image data from the camera and the gaze sequences from the monitoring system, implement a machine learning model comprising a convolutional encoder-decoder neural network configured to process the image data and a side-channel configured to inject the gaze sequences into a decoder stage of the convolutional encoder-decoder neural network, generate, with the machine learning model, a gaze probability density heat map, and generate, with the machine learning model, an attended awareness heat map.

Cognitive heat map: a model for driver situational awareness

A method for prioritizing occlusion information is presented. The method includes determining, at a first time period, a first sensor's view of a spatial area is occluded. The method also includes observing, at a second time period, the spatial area. The method further includes determining a level of risk associated with the spatial area based on the observation. The method still further includes prioritizing transmission of the occlusion information corresponding to the spatial area based on the determined level of risk and transmitting the occlusion information corresponding to the spatial area based on the priority.

Systems and methods described herein relate to estimating risk associated with a vehicular maneuver. One embodiment acquires a geometric representation of an intersection including a lane in which a vehicle is traveling and at least one other lane; discretizes the at least one other lane into a plurality of segments; determines a trajectory along which the vehicle will travel; estimates a probability density function for whether a road agent external to the vehicle is present in the respective segments; estimates a traffic-conflict probability of a traffic conflict in the respective segments conditioned on whether an external road agent is present; estimates a risk associated with the vehicle following the trajectory by integrating a product of the probability density function and the traffic-conflict probability over the at least one other lane and the plurality of segments; and controls operation of the vehicle based, at least in part, on the estimated risk.

Systems and methods for estimating the risk associated with a vehicular maneuver

Systems and methods to systematically identify and collect operational vehicle data for selective transmission to a non-local storage location for further analysis and use in training autonomous and semi-autonomous vehicles are provided. The systems and methods provided overcome limitations in storage and transmission of collected data by selectively archiving only that collected driving data warranting further analysis.

System and method for collection of performance data by a vehicle

A method of providing dynamic and variable learning for an ego vehicle by determining and using most-trustworthy inputs includes determining, based on ambient conditions of an environment of the ego vehicle, a level of trustworthiness of sensor values obtained from one or more ego vehicle sensors. The method also includes determining a level of confidence in an accuracy of an output of a subsystem of the ego vehicle. The output of the subsystem is based on the sensor values. The level of confidence is based on the level of trustworthiness of the sensor values.

Dynamic and variable learning by determining and using most-trustworthy inputs

A system and method for generating a predicted vehicle trajectory includes a generative adversarial network configured to receive a trajectory vector of a target vehicle and generate a set of latent state vectors using the received trajectory vector and an artificial neural network. The latent state vectors each comprise a high-level sub-vector, Z. The GAN enforces Zto be correlated to an annotation coding representing semantic categories of vehicle trajectories. The GAN selects a subset, from the set of latent state vectors, using farthest point sampling and generates a predicted vehicle trajectory based on the selected subset of latent state vectors.

Methods and systems for diversity-aware vehicle motion prediction via latent semantic sampling

Systems, vehicles and methods for determining wrong direction driving are disclosed. In one embodiment, a system for determining a vehicle traveling in a wrong direction includes one or more sensors that produce sensor data, one or more processors, and one or more non-transitory computer-readable medium storing computer readable-instructions. When the computer-readable instructions are executed by the one or more processors, the computer-readable instructions cause the one or more processors to determine one or more lanes within a roadway using the sensor data, determine a direction of travel of the one or more lanes using the sensor data, and identify a non-compliant vehicle traveling in a direction in the one or more lanes that is different from the determined direction of travel in the one or more lanes.

Systems, methods and vehicles for determining wrong direction driving

A method for controlling a driving behavior performed by a first agent includes navigating, by the first agent, according to a trajectory and a velocity. The method also includes receiving, from a second agent, a risk identification message identifying a third agent as a potential risk based on the third agent performing a behavior associated with a probability that is less than a threshold. The method further includes autonomously engaging a defensive driving mode in response to receiving the risk identification message. The method still further includes adjusting one or both of the trajectory or the velocity in response to autonomously engaging the defensive driving mode.

Risk prediction on a peer-to-peer network

Systems and methods for parallel autonomy of a vehicle are disclosed herein. One embodiment receives input data, the input data including at least one of sensor data and structured input data; encodes the input data into an intermediate embedding space using a first neural network; inputs the intermediate embedding space to a first behavior model and a second behavior model, the first behavior model producing a first behavior output, the second behavior model producing a second behavior output; combines the first behavior output and the second behavior output using an ideal-behavior model to produce an ideal behavior for the vehicle; and controls one or more aspects of operation of the vehicle based, at least in part, on the ideal behavior.

Systems and methods for parallel autonomy of a vehicle

An embodiment takes the form of a vehicle that generates a data collection configuration for one or more vehicle sensors of a vehicle based on an estimated information gain to a neural network were the vehicle to provision the neural network with notional sensor data, and based on a vehicle resource consumption by the vehicle were the vehicle to provision the neural network with the notional sensor data. The notional sensor data comprises sensor data that would be collected from a given sensor among the vehicle sensors according to a respective sensor configuration of the given sensor. The vehicle collects sensor data from the vehicle sensors according to the generated data collection configuration.

Methods and systems for collecting sensor data according to a data collection configuration based on an estimated information gain

An embodiment takes the form of a prediction system that obtains a respective indicated similarity of trajectories in each of a plurality of combinations of observed vehicle trajectories. The prediction system, for each of the combinations, calculates a distance between respective intermediate representations, generated by a neural network, of the trajectories in the combination, and performs a training of the neural network based on a calculated loss between the distance and the indicated similarity of the trajectories in the combination. The respective calculated losses converge after performing respective trainings for the combinations. The prediction system creates an intermediate space of respective intermediate representations, generated by the neural network, of observed vehicle trajectories, and predicts a trajectory of a road agent based on a distance between a given intermediate representation in the intermediate space and an intermediate representation of an observed trajectory of the road agent generated by the neural network.

Methods and systems for predicting a trajectory of a road agent based on an intermediate space

Systems and methods for generating driving recommendations are disclosed herein. One embodiment divides automatically a roadway into a plurality of lane-level cells; generates a graph network that represents the plurality of lane-level cells; gathers information pertaining to one or more detected road agents; projects onto the graph network the gathered information pertaining to the one or more detected road agents to update a current status of the plurality of lane-level cells; processes the graph network based on the updated current status of the plurality of lane-level cells to predict a future status of the plurality of lane-level cells, the predicted future status including at least occupancy, by a detected road agent, of the respective lane-level cells in the plurality of lane-level cells; and generates a driving recommendation based, at least in part, on the predicted future status of the plurality of lane-level cells.

Systems and methods for generating driving recommendations

A vehicle system includes a steering wheel configured to output an output based on an input from a user, and a controller configured to: determine a target orientation of front wheels of a vehicle based on vehicle environment information, determine whether an orientation of the front wheels of the vehicle based on the output from the steering wheel deviates from the target orientation, disengage the steering wheel from the front wheels in response to determination that the orientation of the front wheels deviates from the target orientation, adjust the orientation of the front wheels to the target orientation and provide a feedback in response to adjusting the orientation of the front wheels to the target orientation.

Growing community of inventors

Cambridge, MA, United States of America

Luke S Fletcher