Harman International Industries, Inc.

Adam Boulanger

Stefan J Marti

Joseph Verbeke

Davide Di Censo

Sven Kratz

Neeka Mansourian

Evgeny Burmistrov

Priya Seshadri

Jaime Elliot Nahman

Mirjana Spasojevic

Joshua Sweers

Mingwen Dong

Embodiments of the present disclosure sets forth a computer-implemented method comprising receiving, from at least one sensor, sensor data associated with an environment, computing, based on the sensor data, a cognitive load associated with a user within the environment, computing, based on the sensor data, an affective load associated with an emotional state of the user, determining, based on both the cognitive load at the affective load, an affective-cognitive load, determining, based on the affective-cognitive load, a user readiness state associated with the user, and causing one or more actions to occur based on the user readiness state.

Affective-cognitive load based digital assistant

Embodiments of the present disclosure sets forth a computer-implemented method comprising obtaining a starting location and a destination location, determining a plurality of routes from the starting location to the destination location, obtaining affective-cognitive load (ACL) data associated with the plurality of routes, selecting a route included in the plurality of routes based on the ACL data, and transmitting the selected route for output.

Affective-cognitive load based navigation

A system for interactions with an autonomous vehicle includes a proprioceptive device; and a controller. The controller is configured to: detect a first condition external to the autonomous vehicle; generate a first proprioceptive output in response to the first condition, the first proprioceptive output changing a shape of the proprioceptive device; receive a first input force from the proprioceptive device; determine a first user input based on the first input force; and, based on the first user input, cause the autonomous vehicle to perform a navigation action.

Shape-shifting control surface for an autonomous vehicle

Embodiments of the present disclosure set forth a computer-implemented method for identifying an object within an environment comprising receiving, via at least one sensor, first sensor data associated with an environment, storing, in a memory, the first sensor data in association with a first scene, and in response to receiving a user request for information associated with the environment, selecting, based on the user request, the first scene, accessing, via the memory, the first sensor data associated with the first scene, and, analyzing the first sensor data to identify a first object included in the first scene, and causing information associated with the first object to be output via at least one output device.

Automatic reference finding in audiovisual scenes

A system may present notifications to an occupant of a vehicle and determine whether the occupant has given attention to the notification. The system oscillates a visual notification at an output frequency. The system captures images of an eye of the occupant to determine an eye pupil oscillation frequency associated with the occupant. The system compares the eye pupil oscillation frequency to the output frequency. The system determines whether attention was given to the notification based on whether the output frequency and the eye pupil oscillation frequency match. If the system determines that attention was not given, the system may cause performance of one or more follow-up operations.

Attention-based notifications

One or more embodiments include an audio processing system for generating an audio scene for an extended reality (XR) environment. The audio processing system determines that a first virtual sound source associated with the XR environment affects a sound in the audio scene. The audio processing system generates a sound component associated with the first virtual sound source based on a contribution of the first virtual sound source to the audio scene. The audio processing system maps the sound component to a first loudspeaker included in a plurality of loudspeakers. The audio processing system outputs at least a first portion of the component for playback on the first loudspeaker.

Mapping virtual sound sources to physical speakers in extended reality applications

One embodiment of the present disclosure sets forth a technique for delivering audio content to a user. The technique includes analyzing audio content to determine a frequency at which to vibrate an effector. The technique further includes configuring one or more haptic output devices to generate haptic output that generates a haptic trap proximate to the effector. The technique further includes configuring the one or more haptic output devices to modify the haptic output to cause the haptic trap to vibrate the effector at the determined frequency. The effector vibrates to transmit the audio content. In one embodiment, the effector vibrates a skull of a user.

Haptic system for delivering audio content to a user

One embodiment of the present disclosure sets forth a technique for determining the location of an object with a haptic transducer device. The technique includes configuring one or more haptic transducer devices to output a ping. The technique further includes configuring the one or more haptic transducer devices to detect a ping echo associated with the ping. The technique further includes identifying a location of an object based on the ping echo. The technique further includes configuring the one or more haptic transducer devices to generate haptic output based on the location of the object.

Echolocation with haptic transducer devices

Embodiments of the present disclosure sets forth a computer-implemented method for playing back media content comprising determining, based on sensor data, a first mental state associated with a user accessing media content, and modifying playback of media content based on the first mental state.

Modifying media content playback based on user mental state

One embodiment of the present disclosure sets forth a technique for generating an interface on a resonance surface. The technique includes determining a resonance pattern corresponding to a set of interface elements. The technique further includes configuring one or more haptic output devices to generate haptic output that creates the resonance pattern on the resonance surface.

Haptic system for actuating materials

A drone speaker system is configured to deploy a fleet of drone speaker units. Each drone speaker unit includes a speaker configured to broadcast acoustic signals and a flight system configured to aerially transport a speaker. The drone speaker system initially generates a spatial map of a location where the drone speaker units are to be deployed. The drone speaker system then identifies suitable perching locations for the drone speaker units. Then, the drone speaker system deploys the fleet of drone speaker units to those perching locations to place one or more speakers. Once positioned in this manner, the speakers can generate a sound field. The drone speaker units may also reconfigure the speakers to achieve different sound fields having varying characteristics.

Drone deployed speaker system

One embodiment of the present disclosure sets forth a technique for delivering audio output and generates haptic sensations on or proximate to objects in an environment via which a user can perceive haptic sensations. The device includes transducer arrays that operate in various modalities, including generating audio output, generating haptic output, and sensing objects in the environment. When generating audio output, the transducer arrays produce ultrasonic waves that demodulate to produce audio output. When generating haptic output, the transducer arrays output directional haptic output to various locations to the environment using constructive and/or destructive interference. When sensing objects in the environment, the transducer arrays generate pings and detect ping echoes reflected off of objects in the environment.

Haptics device for producing directional sound and haptic sensations

A head-worn audio system includes a support frame, a first contact element configured to contact a first portion of a head of a user, a first actuator coupled to the support frame and configured to move the first contact element, a first sensor configured to generate a first sensor signal indicating a first state of the first contact element and a second sensor signal indicating a second state of the first contact element, and a processor. The processor is communicatively coupled to the first actuator and the first sensor and is configured to cause the first actuator to move the first contact element from a first position that corresponds to the first state to a second position that corresponds to the second state, wherein, in the second state, the first contact element is in contact with the head of the user.

Shape-shifting headphones

One embodiment of the present disclosure sets forth a technique for directing a user to modify an aspect of a vehicle via haptic output. The technique includes identifying one or more characteristics of a real-world condition located along a route of a vehicle. The technique further includes determining, based on the one or more characteristics of the real-world condition, a type of haptic sensation to deliver to a user associated with the vehicle. The technique further includes transmitting a first set of controls signals to at least one haptic output device that is proximate to the user, where the at least one haptic output device generates, based on the first set of control signals, haptic output corresponding to they type of haptic sensation.

Growing community of inventors

Palo Alto, CA, United States of America

Adam Boulanger