Google Inc.

Waymo LLC

Aurora Operations, Inc.

Aurora Innovation, Inc.

Other

Gm Global Technolgoy Operations, Inc.

Christopher Paul Urmson

Dmitri A Dolgov

Philip Nemec

Nathaniel Fairfield

Jiajun Zhu

Luis Ricardo Prada Gomez

Sebastian Thrun

Andrew Timothy Szybalski

Michael Steven Montemerlo

Andrew Hughes Chatham

Sterling J Anderson

James Andrew Bagnell

Ethan Eade

Russell Leigh Smith

Warren Smith

Dirk Haehnel

Bartholomeus C Nabbe

David Ian Ferguson

Daniel Trawick Egnor

Anthony Scott Levandowski

Andy Szybalski

Jared Stephen Russell

Hendrik Dahlkamp

Laura Estelle Lindzey

Nirmal Jitendra Patel

Salil Pandit

Ryan Powell

Min Li Chan

Peter Colijn

Ian James Mahon

Hyman Murveit

Wende Zhang

Bradley Templeton

Varsha K Sadekar

J Andrew Bagnell

Andrew Schultz

Jason Leu

Colin Mease

Implementations described herein relate to enabling natural language communications with an autonomous vehicle. In some implementations, processor(s) of a system can initiate and conduct a conversation with a remote communication participant that is located remotely from the autonomous vehicle whereas, in additional or alternative implementations, the processor(s) can answer an incoming electronic communication and conduct a conversation with a remote communication participant that is located remotely from the autonomous vehicle. In other additional or alternative implementations, the processor(s) can also conduct conversations with a local communication participant that is located proximate to the autonomous vehicle. Notably, the processor(s) can be implemented locally at the autonomous vehicle or remotely from the autonomous vehicle (e.g., at a remote server).

Natural language communications with an autonomous vehicle

Generative models may be used in the detection and/or handling of various events encountered by an autonomous vehicle during its operation. In addition, detected events may may be used in connection with various purposes external from the autonomous vehicle, e.g., for the purpose of surfacing various environmental conditions, events, or other circumstances to a fleet or a third party, for managing vehicle logs and/or for managing teleassist sessions between an autonomous vehicle and a remote teleassist system.

Autonomous vehicle anomalous event detection

Methods and systems are disclosed for cross-validating a second sensor with a first sensor. Cross-validating the second sensor may include obtaining sensor readings from the first sensor and comparing the sensor readings from the first sensor with sensor readings obtained from the second sensor. In particular, the comparison of the sensor readings may include comparing state information about a vehicle detected by the first sensor and the second sensor. In addition, comparing the sensor readings may include obtaining a first image from the first sensor, obtaining a second image from the second sensor, and then comparing various characteristics of the images. One characteristic that may be compared are object labels applied to the vehicle detected by the first and second sensor. The first and second sensors may be different types of sensors.

Cross-validating sensors of an autonomous vehicle

Determining an instantaneous vehicle characteristic (e.g., at least one yaw rate) of an additional vehicle that is in addition to a vehicle being autonomously controlled, and adapting autonomous control of the vehicle based on the determined instantaneous vehicle characteristic of the additional vehicle. For example, autonomous steering, acceleration, and/or deceleration of the vehicle can be adapted based on a determined instantaneous vehicle characteristic of the additional vehicle. In many implementations, the instantaneous vehicle characteristics of the additional vehicle are determined based on data from a phase coherent Light Detection and Ranging (LIDAR) component of the vehicle, such as a phase coherent LIDAR monopulse component and/or a frequency-modulated continuous wave (FMCW) LIDAR component.

Control of autonomous vehicle based on determined yaw parameter(s) of additional vehicle

Aspects of the present disclosure relate generally to limiting the use of an autonomous or semi-autonomous vehicle by particular occupants based on permission data. More specifically, permission data may include destinations, routes, and/or other information that is predefined or set by a third party. The vehicle may then access the permission data in order to transport the particular occupant to the predefined destination, for example, without deviation from the predefined route. The vehicle may drop the particular occupant off at the destination and may wait until the passenger is ready to move to another predefined destination. The permission data may be used to limit the ability of the particular occupant to change the route of the vehicle completely or by some maximum deviation value. For example, the vehicle may be able to deviate from the route up to a particular distance from or along the route.

Systems and methods for vehicles with limited destination ability

A method and apparatus are provided for optimizing one or more object detection parameters used by an autonomous vehicle to detect objects in images. The autonomous vehicle may capture the images using one or more sensors. The autonomous vehicle may then determine object labels and their corresponding object label parameters for the detected objects. The captured images and the object label parameters may be communicated to an object identification server. The object identification server may request that one or more reviewers identify objects in the captured images. The object identification server may then compare the identification of objects by reviewers with the identification of objects by the autonomous vehicle. Depending on the results of the comparison, the object identification server may recommend or perform the optimization of one or more of the object detection parameters.

System and method for evaluating the perception system of an autonomous vehicle

A system and method are arranged to provide recommendations to a user of a vehicle. In one aspect, the vehicle navigates in an autonomous mode and the sensors provide information that is based on the location of the vehicle and output from sensors directed to the environment surrounding the vehicle. In further aspects, both current and previous sensor data is used to make the recommendations, as well as data based on the sensors of other vehicles.

System and method of providing recommendations to users of vehicles

A vehicle configured to operate in an autonomous mode may operate a sensor to determine an environment of the vehicle. The sensor may be configured to obtain sensor data of a sensed portion of the environment. The sensed portion may be defined by a sensor parameter. Based on the environment of the vehicle, the vehicle may select at least one parameter value for the at least one sensor parameter such that the sensed portion of the environment corresponds to a region of interest. The vehicle may operate the sensor, using the selected at least one parameter value for the at least one sensor parameter, to obtain sensor data of the region of interest, and control the vehicle in the autonomous mode based on the sensor data of the region of interest.

Modifying the behavior of an autonomous vehicle using context based parameter switching

Determining classification(s) for object(s) in an environment of autonomous vehicle, and controlling the vehicle based on the determined classification(s). For example, autonomous steering, acceleration, and/or deceleration of the vehicle can be controlled based on determined pose(s) and/or classification(s) for objects in the environment. The control can be based on the pose(s) and/or classification(s) directly, and/or based on movement parameter(s), for the object(s), determined based on the pose(s) and/or classification(s). In many implementations, pose(s) and/or classification(s) of environmental object(s) are determined based on data from a phase coherent Light Detection and Ranging (LIDAR) component of the vehicle, such as a phase coherent LIDAR monopulse component and/or a frequency-modulated continuous wave (FMCW) LIDAR component.

Control of autonomous vehicle based on environmental object classification determined using phase coherent LIDAR data

Aspects of the disclosure relate generally to detecting and avoiding blind spots of other vehicles when maneuvering an autonomous vehicle. Blind spots may include both areas adjacent to another vehicle in which the driver of that vehicle would be unable to identify another object as well as areas that a second driver in a second vehicle may be uncomfortable driving. In one example, a computer of the autonomous vehicle may identify objects that may be relevant for blind spot detecting and may determine the blind spots for these other vehicles. The computer may predict the future locations of the autonomous vehicle and the identified vehicles to determine whether the autonomous vehicle would drive in any of the determined blind spots. If so, the autonomous driving system may adjust its speed to avoid or limit the autonomous vehicle's time in any of the blind spots.

Avoiding blind spots of other vehicles

An autonomous vehicle may access portions of a map to maneuver a roadway. The map may be split into one or more levels that represent different regions in space. For example, an overpass may be represented by one level while the road below the overpass may be on a separate level. A vehicle traveling on a particular level may use map data that is associated with that level. Furthermore, if the vehicle travels through a warp zone, it may transition from the current level to a destination level and thus begin to use map data associated with the destination level.

Method and apparatus to transition between levels using warp zones

Aspects of the disclosure relate to generating a speed plan for an autonomous vehicle. As an example, the vehicle is maneuvered in an autonomous driving mode along a route using pre-stored map information. This information identifies a plurality of keep clear regions where the vehicle should not stop but can drive through in the autonomous driving mode. Each keep clear region of the plurality of keep clear regions is associated with a priority value. A subset of the plurality of keep clear regions is identified based on the route. A speed plan for stopping the vehicle is generated based on the priority values associated with the keep clear regions of the subset. The speed plan identifies a location for stopping the vehicle. The speed plan is used to stop the vehicle in the location.

Planning stopping locations for autonomous vehicles

Growing community of inventors

Mountain View, CA, United States of America

Christopher Paul Urmson