Motional Ad LLC

Nutonomy Inc.

Eric Wolff

Karl D Iagnemma

Harshavardhan Ravichandran

Hsun-Hsien Chang

Emilio Frazzoli

Bence Cserna

Elena Corina Grigore

Philipp Robbel

Tung Minh Phan

Noam Weinstein

Freddy Boulton

Junqing Wei

Oscar Olof Beijbom

Shih-Yuan Liu

Tichakorn Wongpiromsarn

Linh My Pham

Yifan Yang

Guchan Ozbilgin

Dmytro S Yershov

Maurilio Di Cicco

Caglayan Dicle

Matthias Sapuan

Alok Sharma

Abhimanyu Singh

Jacob C Cheuvront

Jeong Hwan Jeon

Akshay Bhagat

Katarzyna Anna Marczuk

Sammie Katt

Mauro Aguiar

Andrew J Eick

David Butterworth

Bishwamoy Sinha Roy

Sammy Omari

Sourabh Vora

Alex Hunter Lang

Bassam Helou

Francesco Seccamonte

Nachiket Deo

Mochan Shrestha

Nanxiang Li

Sang Uk Lee

Juraj Kabzan

Cheng Jiang

Dmitry S Yershov

Tianyi Gu

A scene simulation system can use scene data of a scene of a vehicle to generate one or more simulated states and one or more simulated trajectories associated with the one or more simulated states. The system can evaluate the simulated trajectories and select an action for the vehicle based on the evaluation of the simulated trajectories.

Vehicle action selection based on simulated states

Provided are methods for motion prediction in an autonomous vehicle using fused synthetic and camera images. The method can include obtaining data pairs, each of which reflects data corresponding to a synthetic image representing a birds-eye-view of an area around a vehicle and identifying an object, and data corresponding to a camera image depicting the object. A machine learning model can be trained based on the data pairs to result in a trained model that predicts motion of the object within the data pair based on the synthetic image and camera image in the data pair. Systems and computer program products are also provided.

Motion prediction in an autonomous vehicle using fused synthetic and camera images

Provided are methods for predicting agent trajectories, which can include generating a graph corresponding to a map of a scene by encoding map features and agent features as node encodings of the graph and determining a policy for application to outgoing edges of the nodes of the graph. Some methods described also include sampling paths for a target vehicle in the scene according to the policy and predicting a set of trajectories based on the sampled paths traversed by the policy and a sampled latent variable. Systems and computer program products are also provided.

Predicting agent trajectories

Among other things, techniques are described for receiving, from at least one sensor of a vehicle, sensor data associated with a surface along a path to be traveled by a vehicle; using a surface classifier to determine a classification of the surface based on the sensor data; determining, based on the classification of the surface, drivability properties of the surface; planning, based on the drivability properties of the surface, a behavior of the vehicle when driving near the surface or on the surface; and controlling the vehicle based on the planned behavior.

Road surface condition guided decision making and prediction

Among other things, a vehicle drives autonomously on a trajectory through a road network to a goal location based on an automatic process for planning the trajectory without human intervention; and an automatic process alters the planning of the trajectory to reach a target location based on a request received from an occupant of the vehicle to engage in a speed-reducing maneuver.

Processing a request signal regarding operation of an autonomous vehicle

Among other things, we describe systems and method for validating sensor calibration. For validating calibration of a system of sensors having several types of sensors, an object may be configured to have a substantially reflective portion such that the sensors can isolate the substantially reflective portion, and their sensor data can be compared to determine, if the detected locations of the substantially reflective portion by each sensor are aligned. For calibrating a system of sensors, an object having known calibration features can be used and detected by each sensor, and the detected data can be compared to known calibration data associated with the object to determine if each sensor is correctly calibrated.

Systems and methods for validating sensor calibration

Among other things, techniques are described for driver data guided spatial planning. A spatial structure is generated comprising a plurality of nodes connected by edges. At least some of the nodes and edges represent a path to navigate a vehicle from a first point to a second point. Edges of the spatial structure are labeled as useful based on a distance metric. The spatial structure is pruned by removing one or more edges from the spatial structure according to a respective label of the edges, wherein an extent of the removal is based on a predetermined graph size, a predetermined performance, or any combinations thereof to obtain a pruned graph. A path from the first point to the second point on the pruned graph is identified and the vehicle is navigated in accordance with the path from the first point to the second point on the pruned graph.

Driving data guided spatial planning

Enclosed are embodiments for sampling driving scenarios for training machine learning models. In an embodiment, a method comprises: assigning, using at least one processor, a set of initial physical states to a set of objects in a map for a set of simulated driving scenarios, wherein the set of initial physical states are assigned according to one or more outputs of a random number generator; generating, using the at least one processor, the set of simulated driving scenarios in the map using the initial physical states of the objects in the set of objects; selecting, using the at least one processor, samples of the simulated driving scenarios; training, using the at least one processor, a machine learning model using the selected samples; and operating, using a control circuit, a vehicle in an environment using the trained machine learning model.

Driving scenario sampling for training/tuning machine learning models for vehicles

Among other things, a system provides speed behavior planning for vehicles with autonomous driving capabilities.

Speed behavior planning for vehicles

Among other things, techniques are described for screening and monitoring the health of a vehicle user including receiving sensor data produced by a sensor at the vehicle, processing the sensor data to determine at least one health condition of the user of the vehicle, and in response to determining the at least one health condition, executing a vehicle function selected from a plurality of vehicle functions based on the at least one health condition.

Passenger health screening and monitoring

Among other things, techniques are described for predicting how an agent (e.g., a vehicle, bicycle, pedestrian, etc.) will move in an environment based on prior movement, the road network, the surrounding objects and/or other relevant environmental factors. One trajectory prediction technique involves generating a probability map for an agent's movement. Another trajectory prediction technique involves generating a trajectory lattice, for an agent's movement. In addition, a different trajectory prediction technique involves multi-modal regression where a classifier (e.g., a neural network) is trained to classify the probability of a number of (learned) modes such that each model produces a trajectory based on the current input.

Trajectory prediction from precomputed or dynamically generated bank of trajectories

Among other things, sensor signals are received using a vehicle comprising an autonomous driving capability. A risk associated with operating the vehicle is identified based on the sensor signals. The autonomous driving capability is modified in response to the risk. The operation of the vehicle is updated based on the modifying of the autonomous capability.

Risk processing for vehicles having autonomous driving capabilities

Among other things, we describe techniques for adjusting lateral clearance for a vehicle using a multi-dimensional envelope. A trajectory is generated for the vehicle. A multi-dimensional envelope is generated indicating a drivable region for the vehicle and containing the trajectory. One or more objects are identified located along or adjacent to the trajectory. At least one dimension of the generated multi-dimensional envelope is adjusted to adjust a lateral clearance between the vehicle and the identified one or more objects. A control module of the vehicle navigates the vehicle along the multi-dimensional envelope.

Adjusting lateral clearance for a vehicle using a multi-dimensional envelope

Among other things, techniques are described for conditional motion predictions. The techniques include generating, by a planning circuit, a set of candidate trajectories for a vehicle based on possible macro actions by the vehicle, predicting, by the planning circuit and for at least some trajectories in the set of candidate trajectories, a response by a target vehicle to the respective trajectory and a probability of the response by the target vehicle, selecting, by the planning circuit, a trajectory from the set of candidate trajectories based at least in part on the response by the target vehicle to the trajectory, the probability of the response by the target vehicle, and characteristics of the trajectory, and operating, by a control circuit, the vehicle based on the selected trajectory.

Conditional motion predictions

Growing community of inventors

Cambridge, MA, United States of America

Eric Wolff