Harman International Industries, Inc.

Google Inc.

Joseph Verbeke

Stefan J Marti

Adam Boulanger

Sven Kratz

Davide Di Censo

Evgeny Burmistrov

Priya Seshadri

Neeka Mansourian

Jaime Elliot Nahman

Mirjana Spasojevic

Joshua Sweers

Carsten C Schwesig

Hakim K Raja

Daniel Timothy Pye

Dmitry Aleksandrovich Yashunin

Roman Dmitrievich Vlasov

Mingwen Dong

Tamir Igorevich Baydasov

Embodiments of the present disclosure sets forth a computer-implemented method comprising receiving, from at least one sensor, sensor data associated with an environment, computing, based on the sensor data, a cognitive load associated with a user within the environment, computing, based on the sensor data, an affective load associated with an emotional state of the user, determining, based on both the cognitive load at the affective load, an affective-cognitive load, determining, based on the affective-cognitive load, a user readiness state associated with the user, and causing one or more actions to occur based on the user readiness state.

Affective-cognitive load based digital assistant

Techniques for auditory augmented reality using selective noise cancellation include receiving an input signal capturing an ambient auditory environment; separating the input signal into a set of audio signals that includes first and second component signals; and in response to generating the set of audio signals: generating a context-sensitive user interface that displays a plurality of first controls for modifying the first component signal and a plurality of second controls for modifying the second component signal, the plurality of first controls being independent from the plurality of second controls; receiving, from a user, a selection to remove the first component signal; and in response to receiving the selection: removing the first component signal from the set of audio signals to generate a modified set of audio signals that includes the second component signal; and driving an audio output device to generate sound based on the modified set of audio signals.

Auditory augmented reality using selective noise cancellation

Methods and systems are provided for facial detection techniques for image processing neural networks. In one example, a method may include collecting multi-channel outputs of a set of context modules, providing them to both a face detection head and a landmark localization head of the neural network. The face detection head may then generate bounding boxes which are also provided to the landmark localization head. Based on the output of the context modules and the bounding boxes, the landmark localization head may provide an output including a set of landmark indicators.

Multi-task face detector and landmark detector

A system includes an internal bidirectional communication system configured for communication with a first person disposed in a vehicle; an internal-facing intelligent virtual assistant communicatively coupled to the internal bidirectional communication system; an external bidirectional communication system configured for communication with a second person disposed outside the vehicle; an external-facing intelligent virtual assistant communicatively coupled to the external bidirectional communication system; and a conversation agent that controls information exchanged between the internal-facing intelligent virtual assistant and the external-facing intelligent virtual assistant.

Virtual conversation agent for controlling multiple vehicular intelligent virtual assistants

In various embodiments, a communication application selectively modifies sounds associated with a selected location or entity in one or more images. In operation, the communication application receives an image of an environment of a first device and an audio signal associated with the environment. The communication application receives first user input selecting a location in the image, and modifies a first sound included in the audio signal based on the selected location in the image, where at least a portion of the audio signal is unmodified.

Selective sound modification for video communication

In various embodiments, a sound modification application selectively modifies one or more sounds included in one or more audio signals. In operation, the sound modification application determines classifications associated with multiple sounds included in one or more audio signals. The sound modification application selects a first frequency sub-band of a first sound included in the multiple sounds based on a first classification associated with the first sound. The sound modification application then modifies the first frequency sub-band of the first sound, without modifying at least a second frequency sub-band of the first sound, to generate a modified audio signal.

Sound modification based on frequency composition

A system for interacting with an autonomous vehicle includes a sensor included in the autonomous vehicle and configured to generate sensor data corresponding to a projected hailing area; a projection system included in the autonomous vehicle and configured to generate the projected hailing area on a surface proximate the autonomous vehicle; and a processor included in the autonomous vehicle and configured to execute instructions to: analyze the sensor data to detect the person within the projected hailing area; and in response to detecting the person within the projected hailing area, cause an acknowledgment indicator to be outputted.

Autonomous vehicle interaction system

Various embodiments set forth systems and techniques for explaining domain-specific terms detected in a media content stream. The techniques include detecting a speech portion included in an audio signal; determining that the speech portion comprises a domain-specific term; determining an explanatory phrase associated with the domain-specific term; and integrating the explanatory phrase associated with the domain-specific term into playback of the audio signal.

Techniques for detecting and processing domain-specific terminology

Embodiments of the present disclosure set forth a computer-implemented method comprising receiving, from at least one sensor, sensor data associated with an environment, generating, based on the sensor data, a set of lane change data values associated with positions of at least two vehicles relative to a first lane position in the environment, determining, based on the set of lane change data values, a collision risk value associated with the at least two vehicles attempting to occupy the first lane position, and generating, based on the collision risk value, an output signal to a first vehicle included in the at least two vehicles.

Collision avoidance system for vehicle interactions

A context-aware safety device includes a wireless transceiver, a memory storing an application, and one or more processors. When executing the application, the one or more processors are configured to determine a context of a safety device, configure an alert based on the determined context, and broadcast the configured alert using the wireless transceiver.

Context aware safety alerts

In various embodiments, while a self-driving model operates a vehicle, a user monitoring subsystem acquires sensor data associated with a user of the vehicle and a vehicle observation subsystem acquires sensor data associated with the vehicle. The user monitoring subsystem computes values for a psychological metric based on the sensor data associated with the user. Based on the values for the psychological metric, a feedback application determines a description of the user over a first time period. The feedback application generates a dataset based on the description and the sensor data associated with the vehicle. Subsequently, a training application performs machine learning operation(s) on the self-driving model based on the dataset to generate a modified self-driving model. Advantageously, the dataset enables the training application to automatically modify the self-driving model to account for the impact different driving actions have on the user.

Techniques for customizing self-driving models

Embodiments of the present disclosure sets forth a computer-implemented method comprising obtaining a starting location and a destination location, determining a plurality of routes from the starting location to the destination location, obtaining affective-cognitive load (ACL) data associated with the plurality of routes, selecting a route included in the plurality of routes based on the ACL data, and transmitting the selected route for output.

Affective-cognitive load based navigation

A system for interactions with an autonomous vehicle includes a proprioceptive device; and a controller. The controller is configured to: detect a first condition external to the autonomous vehicle; generate a first proprioceptive output in response to the first condition, the first proprioceptive output changing a shape of the proprioceptive device; receive a first input force from the proprioceptive device; determine a first user input based on the first input force; and, based on the first user input, cause the autonomous vehicle to perform a navigation action.

Shape-shifting control surface for an autonomous vehicle

A system includes a sensing module configured to collect physiological information from a user; an audio input device configured to capture auditory content that is in an environment proximate the user; and one or more processors configured to: determine that a user has entered a state of inwardly focused attention based on first physiological information collected from the user; and in response, record the auditory content captured by the audio input device.

Daydream-aware information recovery system

A virtual private assistant (VPA) is configured to analyze various types of input that indicate one or more behaviors associated with a user and to determine the emotional state of the user based on the input. The VPA also determines one or more operations to perform on behalf of the user based on the input and the determined emotional state. The VPA then executes the one or more operations and synthesizes an output based on the emotional state of the user and the one or more operations. The synthesized output includes one or more semantic components and one or more emotional components derived from the emotional state of the user. The VPA observes the behavior of the user in response to the synthesized output and then implements various modifications, based on the observed behavior, to improve the effectiveness of future interactions with the user.

Growing community of inventors

San Francisco, CA, United States of America

Joseph Verbeke