Ford Global Technolgoies, LLC

Zoox, Inc.

Linjun Zhang

Helen Elizabeth Kourous-Harrigan

Codrin Cionca

Lu Xu

Marin Kobilarov

Juan Enrique Castorena Martinez

Jeffrey Thomas Remillard

Mostafa Parchami

Ankit Girish Vora

Gintaras Vincent Puskorius

Kai Zhenyu Wang

Timothy Caldwell

John Anthony Lockwood

Gowtham Garimella

Samir Parikh

Zhen Zhao

Yunpeng Pan

Benjamin Isaac Mattinson

Xianan Huang

Ke Sun

Joseph Lorenzetti

Chunlei Dai

Helen E Kourous

Yousef Ali Emam

Xiaobo Ren

Woodrow Zhouyuan Wang

A relevance filter may determine to use additional computing resources to generate predicted trajectory(ies) for a subset object(s) detected by a vehicle that depend on a candidate action for controlling the vehicle. The relevance filter may determine the subset by a memory storage and parallel computing technique that includes determining a set of vehicle states associated with a path for controlling the vehicle; determining a set of object states associated with a predicted trajectory of an object; determining a set of interaction scores based on the set of vehicle states and the set of object states; determining, based on the set of interaction scores, a trajectory importance score associated with the predicted trajectory; determining, based on the trajectory importance score, an object importance score associated with the object; and controlling the vehicle based at least in part on the object importance score and/or a new predicted trajectory for the object.

Parallel processing filter for detecting object relevance to vehicle operation planning

Systems and techniques for whether to associate predicted trajectories of objects detected in an environment with candidate vehicle trajectories are described. A vehicle traversing an environment may have several candidate actions that may be used to control the vehicle. Predicted object trajectories may be excluded from further processing based on a relevancy score determination. Various factors may be evaluated to determine the relevancy score associated with a predicted object trajectory and candidate vehicle action pair that indicated the impact of the predicted object trajectory and candidate vehicle action. A resultant trajectory may be determined using only those relevant predicted object trajectories.

Object relevance filtering

Techniques for storing predicted state using a linear array and for querying the linear array to evaluate a proposed trajectory are described herein. In some cases, a first processing unit (e.g., a central processing unit (CPU)) determines prediction data associated with an object in the vehicle environment and sends the prediction data to a second processing unit (e.g., a graphics processing unit (GPU)). In some cases, after the second processing unit receives the prediction data, the second processing unit stores the prediction data using a linear array. In some cases, the second processing unit can use the linear array to evaluate a proposed trajectory for the vehicle and determine whether to validate the proposed trajectory for the vehicle. In some cases, based on the validation of the proposed trajectory by the second processing unit, the vehicle is controlled based on the proposed trajectory.

Evaluating a vehicle trajectory using a linear array query

A machine-learned architecture for determining whether an object is relevant to a vehicle's action planning may comprise a convolutional neural network, graph neural network, and/or multi-layer perceptron that may determine a relevance score associated with an object that indicates indicating whether an object is likely to impact operation(s) of a vehicle. In some examples, the machine-learned architecture may use scene information and/or an object track to determine the relevance score.

Machine-learned model for detecting object relevance to vehicle operation planning

Techniques for determining relevant objects in an environment around a vehicle are discussed herein. Sensor data can be captured and input to a model to determine predicted object trajectories and associated probabilities. For a candidate trajectory for the vehicle to follow, regions of overlap between the vehicle trajectory and the predicted object trajectories can be determined. An interaction graph can be generated representing interactions between the candidate trajectory and the predicted object trajectories. The interaction graph can be sampled based on the probabilities and/or other characteristics. Samples from the interaction graph can be used to determine relevant objects in a driving scenario for a vehicle traversing an environment. Based on the sample, the vehicle may generate a relevancy score associated with the sample. The vehicle may determine that a number of the most relevant sample scenarios may be sent to a prediction component and control of the vehicle.

Identifying relevant objects within an environment

Techniques for determining a vehicle trajectory that causes a vehicle to navigate in an environment relative to one or more objects are described herein. For example, the techniques may include a computing device determining a decision tree having nodes to represent different object intents and/or nodes to represent vehicle actions at a future time. A tree search algorithm can search the decision tree to evaluate potential interactions between the vehicle and the one or more objects over a time period, and output a vehicle trajectory for the vehicle. The vehicle trajectory can be sent to a vehicle computing device for consideration during vehicle planning, which may include simulation.

Trajectory prediction based on a decision tree

This disclosure is directed to techniques for identifying relevant objects within an environment. For instance, a vehicle may use sensor data to determine a candidate trajectory associated with the vehicle and a predicted trajectory associated with an object. The vehicle may then use the candidate trajectory and the predicted trajectory to determine an interaction between the vehicle and the object. Based on the interaction, the vehicle may determine a time difference between when the vehicle is predicted to arrive at a location and when the object is predicted to arrive at the location. The vehicle may then determine a relevance score associated with the object using the time difference. Additionally, the vehicle may determine whether to input object data associated with the object into a planner component based on the relevance score. The planner component determines one or more actions for the vehicle to perform.

A sensor system is disclosed. The sensor system may comprise a housing; an emitter, carried by the housing, that emits a beam comprising depth-data signals; a beam-distribution adjustment system; and a processor programmed to control the adjustment system by selectively changing an angular distribution of the depth-data signals emitted from the housing.

Beam distribution adjustment for a sensor

A system is described. The system includes: a support; a first emitter coupled to the support, facing radially outwardly, and configured to emit a continuous-wave (CW) beam; and a second emitter coupled to the support, facing radially outwardly, and configured to emit a pulsed beam. The system may determine reflectivity using the first emitter and may determine a range using the second emitter.

Depth sensor

A system includes an infrastructure element having a computer. The computer is programmed to receive one or more communication metrics, and node identification data including node location data, from each of a plurality of stationary communication nodes coverage area. The computer is programmed to generate a communication metrics table that identifies locations of the stationary communication nodes and specifies their respective communication metrics. The computer is programmed, based on the communication metrics table, to adjust a transmission parameter of the infrastructure element, the transmission parameter including at least one of a transmission power or a data throughput rate.

Vehicle-to-infrastructure communication control

A system comprising a computer including instructions to collect object data about each of a plurality of objects within a zone from sensors on a roadside infrastructure unit. The zone includes a road intersection. The object data includes an object type, an object location, an object speed and an object direction of travel for each object. The computer further includes instructions to determine, based on the object data, an availability level for execution of one or more paths through the road intersection by a vehicle wherein the availability level for each of the one or more paths is based on a likelihood of interference between any of the objects and the vehicle as the vehicle traverses the respective path. The computer further includes instructions to transmit an availability message to the vehicle including the availability level for each of the one or more paths.

Determining vehicle path

A computer is programmed to determine a localization of a first vehicle, including location coordinates and an orientation of the first vehicle, based on first vehicle sensor data, and to wirelessly receive localizations of respective second vehicles, wherein a first vehicle field of view at least partially overlaps respective fields of view of each of the second vehicles. The computer is programmed to determine pair-wise localizations for respective pairs of the first vehicle and one of the second vehicles, wherein each of the pair-wise localizations defines a localization of the first vehicle relative to a global coordinate system based on a (a) relative localization of the first vehicle with reference to the respective second vehicle and (b) a second vehicle localization relative to the global coordinate system, and to determine an adjusted localization for the first vehicle that has a minimized sum of distances to the pair-wise localizations.

Self-correcting vehicle localization

A system comprises an infrastructure element including a computer programmed to communicate with a first stationary communication node having a first directional short-wave antenna with a first field of view and a second stationary communication node having a second directional short-wave antenna with a second field of view. The first communication node is located within the second field of view. The computer is programmed to determine a first and a second transmission parameter for the first and second stationary communication node respectively based on received object detection sensor data including object data from a respective field of view of each communication node's directional antenna. Each of the first and second transmission parameters includes a transmission power and/or a data throughput rate. The computer is programmed, based on received communication metrics from the first communication node, to determine a deficiency of the second communication node, and upon determining the deficiency of the second communication node, to actuate the first communication node to provide coverage for the second field of view.

Vehicle-to-infrastructure communication control including node deficiency determination

A computer for a roadside infrastructure element includes a processor and a memory. The memory stores instructions executable by the processor to receive communications from a plurality of vehicles; receive sensor data about an area that includes the roadside infrastructure element; based on the sensor data, determine whether the plurality of vehicles includes a priority vehicle or a spoofing vehicle; and specify to the plurality of vehicles a first action upon determining that the plurality of vehicles includes the priority vehicle, and a second action upon determining that the plurality of vehicles includes a spoofing vehicle.

Priority vehicle management

A system comprises a computer including a processor, and a memory. The memory stores instructions such that the processor is programmed to determine two or more clusters of vehicle operating parameter values from each of a plurality of vehicles at a location within a time. Determining the two or more clusters includes clustering data from the plurality of vehicles based on proximity to two or more respective means. The processor is further programmed to determine a reportable condition when a mean for a cluster representing a greatest number of vehicles varies from a baseline by more than a threshold.

Growing community of inventors

Canton, MI, United States of America

Linjun Zhang