Nvidia Corporation

Ram Ganapathi

Ashok Srinivasan

Mohammed Abdulla Yousuf

Niranjan Avadhanam

Mark Henry Costin

Ambrish Dantrey

Atousa Torabi

Revanth Reddy Nalla

Abhijit Patait

Zoran Nikolic

T Y Chan

Mike Truog

Anshul Jain

David Nister

Gary D Hicok

Karl Friedrich Greb

Jonathan Sweedler

John George Mathieson

Clement Farabet

Michael Truog

Michael Alan Ditty

Kevin Flory

Daniel Stewart Perrin

Tai-Yuen Chan

Dan Hettena

TY Chan

Daniel Perrin

In various examples, audio alerts of emergency response vehicles may be detected and classified using audio captured by microphones of an autonomous or semi-autonomous machine in order to identify travel directions, locations, and/or types of emergency response vehicles in the environment. For example, a plurality of microphone arrays may be disposed on an autonomous or semi-autonomous machine and used to generate audio signals corresponding to sounds in the environment. These audio signals may be processed to determine a location and/or direction of travel of an emergency response vehicle (e.g., using triangulation). Additionally, to identify siren types—and thus emergency response vehicle types corresponding thereto—the audio signals may be used to generate representations of a frequency spectrum that may be processed using a deep neural network (DNN) that outputs probabilities of alert types being represented by the audio data.

Emergency response vehicle detection for autonomous driving applications

This disclosure presents an assisted driving vehicle system, including autonomous, semi-autonomous, and technology assisted vehicles, that can share sensor data among two or more controllers. A sensor can have one communication channel to a controller, thereby saving cabling and circuitry costs. The data from the sensor can be sent from one controller to another controller to enable redundancy and backup in case of a system failure. Sensor data from more than one sensor can be aggregated at one controller prior to the aggregated sensor data being communicated to another controller thereby saving bandwidth and reducing transmission times. The sharing of sensor data can be enabled through the use of a sensor data distributor, such as a converter, repeater, or a serializer/deserializer set located as part of the controller and communicatively coupled to another such device in another controller using a data interface communication channel.

Sharing sensor data between multiple controllers to support vehicle operations

In various examples, audio alerts of emergency response vehicles may be detected and classified using audio captured by microphones of an autonomous or semi-autonomous machine in order to identify travel directions, locations, and/or types of emergency response vehicles in the environment. For example, a plurality of microphone arrays may be disposed on an autonomous or semi-autonomous machine and used to generate audio signals corresponding to sounds in the environment. These audio signals may be processed to determine a location and/or direction of travel of an emergency response vehicle (e.g., using triangulation). Additionally, to identify siren types&#x2014;and thus emergency response vehicle types corresponding thereto&#x2014;the audio signals may be used to generate representations of a frequency spectrum that may be processed using a deep neural network (DNN) that outputs probabilities of alert types being represented by the audio data. The locations, direction of travel, and/or siren type may allow an ego-vehicle or ego-machine to identify an emergency response vehicle and to make planning and/or control decisions in response.

Autonomous driving is one of the world's most challenging computational problems. Very large amounts of data from cameras, RADARs, LIDARs, and HD-Maps must be processed to generate commands to control the car safely and comfortably in real-time. This challenging task requires a dedicated supercomputer that is energy-efficient and low-power, complex high-performance software, and breakthroughs in deep learning AI algorithms. To meet this task, the present technology provides advanced systems and methods that facilitate autonomous driving functionality, including a platform for autonomous driving Levels 3, 4, and/or 5. In preferred embodiments, the technology provides an end-to-end platform with a flexible architecture, including an architecture for autonomous vehicles that leverages computer vision and known ADAS techniques, providing diversity and redundancy, and meeting functional safety standards. The technology provides for a faster, more reliable, safer, energy-efficient and space-efficient System-on-a-Chip, which may be integrated into a flexible, expandable platform that enables a wide-range of autonomous vehicles, including cars, taxis, trucks, and buses, as well as watercraft and aircraft.

Systems and methods for safe and reliable autonomous vehicles

In a self-driving autonomous vehicle, a controller architecture includes multiple processors within the same box. Each processor monitors the others and takes appropriate safe action when needed. Some processors may run dormant or low priority redundant functions that become active when another processor is detected to have failed. The processors are independently powered and independently execute redundant algorithms from sensor data processing to actuation commands using different hardware capabilities (GPUs, processing cores, different input signals, etc.). Intentional hardware and software diversity improves fault tolerance. The resulting fault-tolerant/fail-operational system meets ISO26262 ASIL-D specifications based on a single electronic controller unit platform that can be used for self-driving vehicles.

Method of using a single controller (ECU) for a fault-tolerant/fail-operational self-driving system

In a self-driving autonomous vehicle, a controller architecture includes multiple processors within the same box. Each processor monitors the others and takes appropriate safe action when needed. Some processors may run dormant or low priority redundant functions that become active when another processor is detected to have failed. The processors are independently powered and independently execute redundant algorithms from sensor data processing to actuation commands using different hardware capabilities (GPUs, processing cores, different input signals, etc.). Intentional hardware and software diversity improves fault tolerance. The resulting fault-tolerant/fail-operational system meets ISO26262 ASIL D specifications based on a single electronic controller unit platform that can be used for self-driving vehicles.

This disclosure presents an assisted driving vehicle system, including autonomous, semi-autonomous, and technology assisted vehicles, that can share sensor data among two or more controllers. A sensor can have one communication channel to a controller, thereby saving cabling and circuitry costs. The data from the sensor can be sent from one controller to another controller to enable redundancy and backup in case of a system failure. In another embodiment, sensor data from more than one sensor can be aggregated at one controller prior to the aggregated sensor data being communicated to another controller thereby saving bandwidth and reducing transmission times. The sharing of sensor data can be enabled through the use of a sensor data distributor, such as a converter, repeater, or a serializer/deserializer set located as part of the controller and communicatively coupled to another such device in another controller using a data interface communication channel.

Growing community of inventors

San Jose, CA, United States of America

Ram Ganapathi