Apple Inc.

Shai Messingher Lang

Symeon Delikaris Manias

Christopher T Eubank

Jonathan D Sheaffer

Alexandre Da Veiga

Spencer H Ray

Martin E Johnson

Nikolas T Vitt

Daniel K Boothe

Neal D Evans

Peter Victor Jupin

David Michael Cole

Juha O Merimaa

Samir Gehani

Danvin Ruangchan

Travis W Brown

Joshua J Taylor

Jared King

In one implementation, a method of visualizing a combined audio pick-up pattern is performed at a first device in a physical environment, the first device including a display, one or more processors, and non-transitory memory. The method includes determining a first audio pick-up pattern of the first device. The method includes determining one or more second audio pick-up patterns of a respective one or more second devices. The method includes determining a combined audio pick-up pattern of the first device and the one or more second devices based on the first audio pick-up pattern and the one or more second audio pick-up patterns. The method includes displaying, on the display, a representation of the combined audio pick-up pattern. In one implementation, a method of determining an audio emission pattern is performed at a first device at a first location, the first device having a microphone, one or more processors, and non-transitory memory. The method includes obtaining, via the microphone, first audio of a sound source. The method includes receiving, from one or more second devices, one or more second audio of the sound source. The method includes determining one or more second locations of the one or more second devices. The method includes determining an audio emission pattern of the sound source based on the first audio data, the one or more second audio data, and the one or more second locations, wherein the audio emission pattern of the sound source indicates a sound level at various locations relative to the sound source.

Audio capture with multiple devices

Various implementations disclosed herein include devices, systems, and methods that provide video content (e.g., a TV show, a recorded sporting event, a movie, a 3D video, etc.) within a 3D environment using parameters selected based on one or more contextual factors. Such contextual factors may be determined based on an attribute of the content (e.g., its intended purpose or viewing environment), the user (e.g., the user's visual quality, interpupillary distance, etc.), the 3D environment (e.g., current lighting conditions, spatial considerations, etc.), or other context attributes.

Parameter selection for media playback

A method for self-calibrating a sound pickup process that uses a microphone array in a wearable device that also includes a loudspeaker, where the microphone array being in a physical arrangement with respect to the loudspeaker. The method obtains, for each of several microphones of the microphone array, one or more transfer functions that each represent a response of the microphone to sound from a position in an acoustic space. The method determines whether a physical arrangement of the microphone array with respect to the loudspeaker has changed and adjusts the transfer function, for at least one of the microphones of the several microphones, in response to determining that the current physical arrangement of the microphone array with respect to the loudspeaker has changed.

Self-calibrating microphone and loudspeaker arrays for wearable audio devices

Various implementations disclosed herein include devices, systems, and methods that display visual content as part of a 3D environment and add audio corresponding to the visual content. The audio may be spatialized to be from one or more audio source locations within the 3D environment. For example, a video may be presented on a virtual surface within an extended reality (XR) environment while audio associated with the video is spatialized to sound as if it is produced from an audio source location corresponding to that virtual surface. How the audio is provided may be determined based on the position of the viewer (e.g., the user or his/her device) relative to the presented visual content.

Visual content presentation with viewer position-based audio

Each of a plurality of virtual loudspeaker arrays and their channels are produced, based on a corresponding microphone array and microphone signals thereof. Channels of a hallucinated loudspeaker array are determined based on the channels of the plurality of virtual loudspeaker arrays. The plurality of virtual loudspeaker arrays and the hallucinated loudspeaker array share a common geometry and orientation. Spatial audio is rendered based on the channels of the hallucinated loudspeaker array.

Auditory origin synthesis

Various implementations disclosed herein include devices, systems, and methods that that modify audio of played back AV content based on context in accordance with some implementations. In some implementations audio-visual content of a physical environment is obtained, and the audio-visual content includes visual content and audio content that includes a plurality of audio portions corresponding to the visual content. In some implementations, a context for presenting the audio-visual content is determined, and a temporal relationship between one or more audio portions of the plurality of audio portions and the visual content is determined based on the context. Then, synthesized audio-visual content is presented based on the temporal relationship.

Variable audio for audio-visual content

A machine learning model can determine output frequency response at different directions relative to a target audio output format, based on input including frequency response at directions relative to microphones of a capture device. A spatial filter determined based on the output frequency responses is applied to one or more of the microphone signals to map the spatial information from the microphone signals to the target audio output.

Deep-learning based beam forming synthesis for spatial audio

Spatial filters are generated that map response of an audio capture device to head related transfer functions (HRTFs) for different positions of the audio capture device relative to the HRTFs. A current set of spatial filters are determined based on the plurality of spatial filters and a head position of a user. The microphone signals are convolved with the current set of spatial filters, resulting in a left audio channel and right audio channel that form output binaural audio channels. The binaural audio channels can be used to drive speakers of a headphone set to generate sound that is perceived to have a spatial quality. Other aspects are described and claimed.

Growing community of inventors

Santa Clara, CA, United States of America

Shai Messingher Lang