Harman International Industries, Inc.

Evgeny Burmistrov

Stefan J Marti

Joseph Verbeke

Priya Seshadri

Sven Kratz

Adam Boulanger

Neeka Mansourian

Mingwen Dong

Embodiments of the present disclosure sets forth a computer-implemented method comprising receiving, from at least one sensor, sensor data associated with an environment, computing, based on the sensor data, a cognitive load associated with a user within the environment, computing, based on the sensor data, an affective load associated with an emotional state of the user, determining, based on both the cognitive load at the affective load, an affective-cognitive load, determining, based on the affective-cognitive load, a user readiness state associated with the user, and causing one or more actions to occur based on the user readiness state.

Affective-cognitive load based digital assistant

A system includes an internal bidirectional communication system configured for communication with a first person disposed in a vehicle; an internal-facing intelligent virtual assistant communicatively coupled to the internal bidirectional communication system; an external bidirectional communication system configured for communication with a second person disposed outside the vehicle; an external-facing intelligent virtual assistant communicatively coupled to the external bidirectional communication system; and a conversation agent that controls information exchanged between the internal-facing intelligent virtual assistant and the external-facing intelligent virtual assistant.

Virtual conversation agent for controlling multiple vehicular intelligent virtual assistants

Various embodiments set forth systems and techniques for explaining domain-specific terms detected in a media content stream. The techniques include detecting a speech portion included in an audio signal; determining that the speech portion comprises a domain-specific term; determining an explanatory phrase associated with the domain-specific term; and integrating the explanatory phrase associated with the domain-specific term into playback of the audio signal.

Techniques for detecting and processing domain-specific terminology

Embodiments of the present disclosure set forth a computer-implemented method comprising receiving, from at least one sensor, sensor data associated with an environment, generating, based on the sensor data, a set of lane change data values associated with positions of at least two vehicles relative to a first lane position in the environment, determining, based on the set of lane change data values, a collision risk value associated with the at least two vehicles attempting to occupy the first lane position, and generating, based on the collision risk value, an output signal to a first vehicle included in the at least two vehicles.

Collision avoidance system for vehicle interactions

Embodiments of the present disclosure set forth a computer-implemented method comprising detecting an initial phrase portion included in a first auditory signal generated by a user, identifying, based on the initial phrase portion, a supplemental phrase portion that complements the initial phrase portion to form a complete phrase, and providing a command signal that drives an output device to generate an audio output corresponding to the supplemental phrase portion.

Techniques for dynamic auditory phrase completion

Techniques for modifying sound based on a direction of interest include determining a direction of interest associated with a user; receiving a set of audio signals associated with the direction of interest; determining one or more salient frequency bands within the set of audio signals; in response to receiving a command from the user to enhance or suppress the set of audio signals, enhancing or suppressing a portion of the set of audio signals corresponding to the one or more salient frequency bands to create a modified set of audio signals; and outputting the modified set of audio signals.

Sound modification based on direction of interest

Disclosed embodiments include techniques for providing alerts to a driver of a vehicle. The techniques include detecting a condition that exceeds a threshold hazard potential; detecting a predetermined gesture of a driver, the predetermined gesture indicating that the driver has acknowledged the detected condition; in response to detecting the predetermined gesture, reducing an urgency level for alerting the driver of the detected condition; and determining whether to issue an alert to the driver based on the reduced urgency level.

Techniques for detecting acknowledgment from a driver of a vehicle

A system may present notifications to an occupant of a vehicle and determine whether the occupant has given attention to the notification. The system oscillates a visual notification at an output frequency. The system captures images of an eye of the occupant to determine an eye pupil oscillation frequency associated with the occupant. The system compares the eye pupil oscillation frequency to the output frequency. The system determines whether attention was given to the notification based on whether the output frequency and the eye pupil oscillation frequency match. If the system determines that attention was not given, the system may cause performance of one or more follow-up operations.

Attention-based notifications

Embodiments of the present disclosure sets forth a computer-implemented method comprising obtaining sensor data associated with a driver of a vehicle, determining a state associated with the driver based on the sensor data, and transmitting the state associated with the driver to a teleconference session, wherein the driver is at least one of a participant or an invitee to the teleconference session.

Conferencing based on driver state and context

One or more embodiments include techniques for providing an alert, based on monitoring of a driver, via a pedestrian-based vehicle-to-pedestrian (V2P) system associated with a pedestrian. A pedestrian-based V2P system receives, from a vehicle-based V2P system, data related to an awareness level of the driver of the vehicle. The data may include data specifying the awareness level of a driver with respect to a pedestrian, metrics that are determinative of the awareness level of the driver, raw measurement data from the internally facing sensors and/or externally facing sensors, alerts in digitized audio or other suitable format to transmit to the pedestrian, and/or the like. The pedestrian-based V2P system generates an alert based on at least one of a location of the vehicle and the awareness level of the driver of the vehicle. The pedestrian-based V2P system transmits the alert to an output device.

Techniques for generating vehicle-to-pedestrian (V2P) communications based on driver monitoring

Embodiments of the present disclosure sets forth a computer-implemented method comprising acquiring image information associated with an environment, acquiring, from one or more sensors, sensor data associated with a gaze of a user, determining a source of interest based on the image information and the sensor data, processing a set of audio signals associated with the environment based on the image information to identify an audio signal associated with the source of interest, enhancing the audio signal associated with the source of interest relative to other audio signals in the set of audio signals, and outputting the enhanced audio signal associated with the source of interest to the user.

Audio-visual sound enhancement

In various embodiments, an alert personalization application personalizes in-vehicle alerts. The alert personalization application estimates an increase in a negative emotion associated with an occupant of a vehicle based on at least one characteristic of the occupant. Subsequently, the alert personalization application determines that the increase is associated with a negative reaction to an in-vehicle alert that has a first alert characteristic. In response, the alert personalization application determines that the first alert characteristic should be replaced with a second alert characteristic. The alert personalization causes another in-vehicle alert to be generated that has the second alert characteristic instead of the first alert characteristic. Advantageously, personalizing alert characteristics that influence how and/or when in-vehicle alerts are generated can increase the effectiveness of in-vehicle alerts and can improve driving safety.

Techniques for automatically reducing annoyance levels of drivers when using driver monitoring systems

Disclosed embodiments include techniques for detecting an acknowledgement from a driver of a vehicle. A driver state and reaction tracking system determines that a detected condition exceeds a threshold hazard potential with the vehicle. The driver state and reaction tracking system detects a gesture, by a driver, associated with the detected condition. The driver state and reaction tracking system, in response to detecting the gesture, reduces an urgency level for alerting the driver of the detected condition to generate a reduced urgency level. The driver state and reaction tracking system determines whether to issue an alert to the driver based on the reduced urgency level. The driver state and reaction tracking system may transmit an acknowledgement to other systems associated with pedestrians, bicyclists, motorcyclists, semiautonomous vehicles, manually driven vehicles and/or the like. to confirm that the driver has seen the pedestrian, bicyclist, motorcyclist, or vehicle associated with the other system.

Embodiments of the present disclosure set forth a computer-implemented method comprising determining a first direction of interest associated with a user, receiving a set of audio signals associated with the first direction of interest, determining a first dominant frequency band within the set of audio signals, modifying a first portion of the set of audio signals corresponding to the first dominant frequency band, and outputting the modified set of audio signals.

Growing community of inventors

Saratoga, CA, United States of America

Evgeny Burmistrov