Zoox, Inc.

Gopro, Inc.

William Anthony Silva

Jefferson Bradfield Packer

Timothy Caldwell

Abishek Krishna Akella

Max Gier

Dragomir Dimitrov Anguelov

Benjamin Isaac Zwiebel

Rick Zhang

Juhana Kangaspunta

Marin Kobilarov

Stephen Yui

Ashwani Arya

Dan Xie

Mehran Ferdowsi

Zhenqi Huang

Vikrant More

Janek Hudecek

Andrew E Beller

Matthew Van Heukelom

Moritz Harmel

Chuang Wang

Gowtham Garimella

Marc Wimmershoff

Lichao Ma

A vehicle computing system may implement techniques to control a vehicle to avoid collisions between the vehicle and agents (e.g., dynamic objects) in an environment. The techniques may include generating a representation of a path of the vehicle through an environment as a polygon. The vehicle computing system may compare the two-dimensional path with a trajectory of an agent determined using sensor data to determine a collision zone between the vehicle and the agent. The vehicle computing system may determine a risk of collision based on predicted velocities and probable accelerations of the vehicle and the agent approaching and traveling through the collision zone. Based at least in part on the risk of collision, the vehicle computing system may cause the vehicle to perform an action.

Collision prediction and avoidance for vehicles

Techniques for determining to modify a trajectory based on an object are discussed herein. A vehicle can determine a drivable area of an environment, capture sensor data representing an object in the environment, and perform a spot check to determine whether or not to modify a trajectory. Such a spot check may include processing to incorporate an actual or predicted extent of the object into the drivable area to modify the drivable area. A distance between a reference trajectory and the object can be determined at discrete points along the reference trajectory, and based on a cost, distance, or intersection associated with the trajectory and the modified area, the vehicle can modify its trajectory. One trajectory modification includes following, which may include varying a longitudinal control of the vehicle, for example, to maintain a relative distance and velocity between the vehicle and the object.

Vehicle trajectory modification for following

This disclosure describes a method of controlling an unmanned aerial vehicle (UAV). The steps of controlling include acquiring images with an image capture device of an unmanned aerial vehicle (UAV). The steps include analyzing the images to determine navigation information of the UAV with a vision-based navigation system. The steps include tracking a position of the UAV with the vision-based navigation system. The steps include controlling rotors of the UAV to prevent deviations in movement from a desired flight path or position of the UAV. The steps include limiting travel or flight of the UAV to a physical region determined by the desired flight path.

Multipoint cable cam system and method

Drive envelope determination is described. In an example, a vehicle can capture sensor data while traversing an environment and can provide the sensor data to computing system(s). The sensor data can indicate agent(s) in the environment and the computing system(s) can determine, based on the sensor data, a planned path through the environment relative to the agent(s). The computing system(s) can also determine lateral distance(s) to the agent(s) from the planned path. In an example, the computing system(s) can determine modified distance(s) based at least in part on the lateral distance(s) and information about the agents. The computing system can determine a drive envelope based on the modified distance(s) and can determine a trajectory in the drive envelope.

Drive envelope determination

Techniques are discussed for controlling a vehicle, such as an autonomous vehicle, based on occluded areas in an environment. An occluded area can represent areas where sensors of the vehicle are unable to sense portions of the environment due to obstruction by another object. An occlusion grid representing the occluded area can be stored as map data or can be dynamically generated. An occlusion grid can include occlusion fields, which represent discrete two- or three-dimensional areas of driveable environment. An occlusion field can indicate an occlusion state and an occupancy state, determined using LIDAR data and/or image data captured by the vehicle. An occupancy state of an occlusion field can be determined by ray casting LIDAR data or by projecting an occlusion field into segmented image data. The vehicle can be controlled to traverse the environment when a sufficient portion of the occlusion grid is visible and unoccupied.

Occlusion aware planning and control

Junction queueing for vehicles is described, in which vehicles may be queued based on arrival time at a junction, position relative to a stopping location at the junction, and/or an amount of time waiting for other vehicles to proceed through the junction (timeout). In some examples, the queue may be first generated based on the arrival times of any other vehicle relative to a particular vehicle generating the queue. The queue may be updated based on arrival times of other vehicles (e.g., after the particular vehicle), whether another vehicle has proceeded out-of-turn (e.g., based on a position at the junction), and/or a timeout for vehicles that wait for others to yield at the junction. In some examples, hysteresis and alterations of the score for safety reasons may alter queue order. The queue may be used to control the particular vehicle to traverse the junction.

Junction queueing

Techniques are discussed for controlling a vehicle, such as an autonomous vehicle, based on occluded areas in an environment. An occluded area can represent areas where sensors of the vehicle are unable to sense portions of the environment due to obstruction by another object or sensor limitation. An occluded region for an object is determined, along with one or more visible regions proximate the occluded region. Entry and/or exit regions may be determined based on known directions of traffic and/or drivable surface boundaries. Based on a threshold speed, the vehicle can designate portions of the occluded region as pseudo-visible. Additionally, if a dynamic object traverses through the occluded region from the entry region, a portion of the occluded region may be considered pseudo-visible. Pseudo-visibility may also be determined based on movement of an occluded area. The vehicle can be controlled to traverse the environment based on the pseudo-visibility.

Determining occupancy of occluded regions

Techniques for generating simulations for evaluating a performance of a controller of an autonomous vehicle are described. A computing system may evaluate the performance of the controller to navigate the simulation and respond to actions of one or more objects (e.g., other vehicles, bicyclists, pedestrians, etc.) in a simulation. Actions of the objects in the simulation may be controlled by the computing system (e.g., by an artificial intelligence) and/or one or more users inputting object controls, such as via a user interface. The computing system may calculate performance metrics associated with the actions performed by the vehicle in the simulation as directed by the autonomous controller. The computing system may utilize the performance metrics to verify parameters of the autonomous controller (e.g., validate the autonomous controller) and/or to train the autonomous controller utilizing machine learning techniques to bias toward preferred actions.

Vehicle controller simulations

This disclosure describes systems and methods for a multipoint cable cam (MPCC) of an aerial vehicle. A method includes operations of receiving user input associated with a predetermined path and correlating the received user input with stored global positioning satellite (GPS) data to generate one or more virtual waypoints along the predetermined path. The method includes processing the one or more virtual waypoints to generate a spline-based flight path. The method may include storing the spline-based flight path and transmitting the spline-based flight path to the aerial vehicle.

A vehicle can traverse an environment along a first region and detect an obstacle impeding progress of the vehicle. The vehicle can determine a second region that is adjacent to the first region and associated with a direction of travel opposite the first region. The vehicle can use a state machine to determine an action (e.g., an oncoming action) to utilize the second region to overtake the obstacle. By comparing a cost to a cost threshold and/or to a cost associated with another action (e.g., a 'stay in lane' action), the vehicle, using the state machine, can determine a target trajectory that traverses through the second region and can traverse the environment based on the target trajectory to avoid, for example, the obstacle in the environment while maintaining a safe distance from the obstacle and/or other entities in the environment.

State machine for obstacle avoidance

Techniques are discussed for controlling a vehicle, such as an autonomous vehicle, based on occluded areas in an environment. An occluded area can represent areas where sensors of the vehicle are unable to sense portions of the environment due to obstruction by another object or sensor limitation. An occluded region for an object is determined by the vehicle as part of an occlusion grid, from the perspective of the vehicle. The vehicle may receive another occlusion grid from another source, such as another vehicle or a remote computing device that stores and distributes occlusion grids. The other occlusion grid may be from a different perspective than the occlusion grid generated by the vehicle, and may include occupancy data for the region that is otherwise occluded from the perspective of the vehicle. The vehicle can be controlled to traverse the environment based on the occupancy data received from the other source.

Sharing occlusion data

Techniques described herein are directed to classifying lanes in an environment of a vehicle for, for example, performing lane handling. In an example, system(s) of a vehicle can determine a signal indicative of a presence of the vehicle in a lane of a drivable surface in an environment within which the vehicle is located. The system(s) can determine, based at least in part on the signal, a classification of the lane as at least one of occupied (an object is at least partially in the lane), unoccupied (no object in the lane), and/or established (e.g., where an object has established a priority in the lane). The system(s) can control the vehicle based at least in part on the classification of the lane to improve safety in scenarios, for example, including merging.

Lane classification for improved vehicle handling

A vehicle safety system within an autonomous or semi-autonomous vehicle may predict and avoid collisions between the vehicle and other moving objects in the environment. The vehicle safety system may determine one or more perturbed trajectories for another object in the environment, for example, by perturbing the state parameters of a perceived trajectory associated with the object. Each perturbed trajectory may be evaluated to determine whether it intersects or potentially collides the planned trajectory of the vehicle. In some examples, the vehicle safety system may aggregate the results of analyses of multiple perturbed trajectories to determine a collision probability and/or additional weights or adjustment factors associated with the collision prediction, and may determine actions for the vehicle to take based on the collision predictions and probabilities.

Vehicle collision avoidance based on perturbed object trajectories

A vehicle can traverse an environment along a first region and detect an obstacle impeding progress of the vehicle. The vehicle can determine a second region that is adjacent to the first region and associated with a direction of travel opposite the first region. A cost can be determined based on an action (e.g., an oncoming action) to utilize the second region to overtake the obstacle. By comparing the cost to a cost threshold and/or to a cost associated with another action (e.g., a 'stay in lane' action), the vehicle can determine a target trajectory that traverses through the second region. The vehicle can traverse the environment based on the target trajectory to avoid, for example, the obstacle in the environment while maintaining a safe distance from the obstacle and/or other entities in the environment.

Growing community of inventors

San Francisco, CA, United States of America

William Anthony Silva