Nvidia Corporation

Atousa Torabi

Niranjan Avadhanam

Sakthivel Sivaraman

Shagan Sah

Ambrish Dantrey

Ram Ganapathi

Revanth Reddy Nalla

Jason Conrad Roche

Abhijit Patait

Anshul Jain

Yuzhuo Ren

Nishant Puri

Sumit Bhattacharya

Shubhadeep Das

Sumit Kumar Bhattacharya

In various examples, audio alerts of emergency response vehicles may be detected and classified using audio captured by microphones of an autonomous or semi-autonomous machine in order to identify travel directions, locations, and/or types of emergency response vehicles in the environment. For example, a plurality of microphone arrays may be disposed on an autonomous or semi-autonomous machine and used to generate audio signals corresponding to sounds in the environment. These audio signals may be processed to determine a location and/or direction of travel of an emergency response vehicle (e.g., using triangulation). Additionally, to identify siren types—and thus emergency response vehicle types corresponding thereto—the audio signals may be used to generate representations of a frequency spectrum that may be processed using a deep neural network (DNN) that outputs probabilities of alert types being represented by the audio data.

Emergency response vehicle detection for autonomous driving applications

Interactions with virtual systems may be difficult when users inadvertently fail to provide sufficient information to proceed with their requests. Certain types of inputs, such as auditory inputs, may lack sufficient information to properly provide a response to the user. Additional information, such as image data, may enable user gestures or poses to supplement the auditory inputs to enable response generation without requesting additional information from users.

Multi-modal sensor fusion for content identification in applications of human-machine interfaces

In various examples, systems and methods are disclosed that accurately identify driver and passenger in-cabin activities that may indicate a biomechanical distraction that prevents a driver from being fully engaged in driving a vehicle. In particular, image data representative of an image of an occupant of a vehicle may be applied to one or more deep neural networks (DNNs). Using the DNNs, data indicative of key point locations corresponding to the occupant may be computed, a shape and/or a volume corresponding to the occupant may be reconstructed, a position and size of the occupant may be estimated, hand gesture activities may be classified, and/or body postures or poses may be classified. These determinations may be used to determine operations or settings for the vehicle to increase not only the safety of the occupants, but also of surrounding motorists, bicyclists, and pedestrians.

In-cabin hazard prevention and safety control system for autonomous machine applications

In various examples, audio alerts of emergency response vehicles may be detected and classified using audio captured by microphones of an autonomous or semi-autonomous machine in order to identify travel directions, locations, and/or types of emergency response vehicles in the environment. For example, a plurality of microphone arrays may be disposed on an autonomous or semi-autonomous machine and used to generate audio signals corresponding to sounds in the environment. These audio signals may be processed to determine a location and/or direction of travel of an emergency response vehicle (e.g., using triangulation). Additionally, to identify siren types&#x2014;and thus emergency response vehicle types corresponding thereto&#x2014;the audio signals may be used to generate representations of a frequency spectrum that may be processed using a deep neural network (DNN) that outputs probabilities of alert types being represented by the audio data. The locations, direction of travel, and/or siren type may allow an ego-vehicle or ego-machine to identify an emergency response vehicle and to make planning and/or control decisions in response.

Systems and methods are disclosed herein for a pedestrian crossing warning system that may use multi-modal technology to determine attributes of a person and provide a warning to the person in response to a calculated risk level to effect a reduction of the risk level. The system may utilize sensors to receive data indicative of a trajectory of a person external to the vehicle. Specific attributes of the person such as age or walking aids may be determined. Based on the trajectory data and the specific attributes, a risk level may be determined by the system using a machine learning model. The system may cause emission of a warning to the person in response to the risk level.

Systems and methods for pedestrian crossing risk assessment and directional warning

Growing community of inventors

Santa Clara, CA, United States of America

Atousa Torabi