Apple Inc.

Symeon Delikaris Manias

Jonathan D Sheaffer

Shai Messingher Lang

Joshua D Atkins

Mehrez Souden

Ante Jukic

Peter A Raffensperger

Martin E Johnson

Alexandre Da Veiga

Juha O Merimaa

Spencer H Ray

Frank M Baumgarte

Christopher T Eubank

Nikolas T Vitt

Daniel K Boothe

Neal D Evans

Matthew S Connolly

Peter Victor Jupin

Sabine Webel

Ronald J Guglielmone, Jr

Gaetan R Lorho

Dipanjan Sen

Sean Anthony Ramprashad

Justin Derry Crosby

John Woodruff

Eric A Allamanche

Ismael Hamad Nawfal

Jason Wung

Ramin Pishehvar

Bonnie W Tom

Emily A Wigley

Jared King

Abhaya Parthy

A multi-radius spherical microphone that includes an inner body defining an inner sphere having an inner radius from a center; a plurality of inner microphones coupled to the inner spherical body and defining an array of inner microphones; an outer body defining an dodecahedron, wherein the inner body and the outer body are concentric about the center; and a plurality of outer microphones coupled to the outer body at respective vertices of the dodecahedron and defining an array of outer microphones, wherein each of the plurality of outer microphones is positioned radially equidistant from the center.

Apple dual concentric spherical microphone array

A device may include a plurality of microphones, and a processor configured to extract a sound and a sound field from microphone signals of the plurality of microphones. The sound may be treated as near-field sound and the sound field may be treated as far-field sound. The device may adjust a strength of the sound based on a strength of the sound field and apply near-field head related transfer functions (HRTFs) to the sound. The device may apply far-field HRTFs to the sound field. The device may combine the near-field applied sound with the far-field applied sound field to generate spatial audio for playback through a plurality of speakers.

Spatial audio using near-field and far-field rendering

In one implementation, a method of visualizing a combined audio pick-up pattern is performed at a first device in a physical environment, the first device including a display, one or more processors, and non-transitory memory. The method includes determining a first audio pick-up pattern of the first device. The method includes determining one or more second audio pick-up patterns of a respective one or more second devices. The method includes determining a combined audio pick-up pattern of the first device and the one or more second devices based on the first audio pick-up pattern and the one or more second audio pick-up patterns. The method includes displaying, on the display, a representation of the combined audio pick-up pattern. In one implementation, a method of determining an audio emission pattern is performed at a first device at a first location, the first device having a microphone, one or more processors, and non-transitory memory. The method includes obtaining, via the microphone, first audio of a sound source. The method includes receiving, from one or more second devices, one or more second audio of the sound source. The method includes determining one or more second locations of the one or more second devices. The method includes determining an audio emission pattern of the sound source based on the first audio data, the one or more second audio data, and the one or more second locations, wherein the audio emission pattern of the sound source indicates a sound level at various locations relative to the sound source.

Audio capture with multiple devices

A method for self-calibrating a sound pickup process that uses a microphone array in a wearable device that also includes a loudspeaker, where the microphone array being in a physical arrangement with respect to the loudspeaker. The method obtains, for each of several microphones of the microphone array, one or more transfer functions that each represent a response of the microphone to sound from a position in an acoustic space. The method determines whether a physical arrangement of the microphone array with respect to the loudspeaker has changed and adjusts the transfer function, for at least one of the microphones of the several microphones, in response to determining that the current physical arrangement of the microphone array with respect to the loudspeaker has changed.

Self-calibrating microphone and loudspeaker arrays for wearable audio devices

Processing of ambience and speech can include extracting from audio signals, ambience and speech signals. One or more spatial parameters can be generated that define spatial characteristics of ambience sound in the one or more ambience audio signals. The primary speech signal, the one or more ambience audio signals, and the spatial parameters can be encoded into one or more encoded data streams. Other aspects are described and claimed.

Separating and rendering voice and ambience signals by offsetting impact of device movements

Various implementations disclosed herein include devices, systems, and methods that display visual content as part of a 3D environment and add audio corresponding to the visual content. The audio may be spatialized to be from one or more audio source locations within the 3D environment. For example, a video may be presented on a virtual surface within an extended reality (XR) environment while audio associated with the video is spatialized to sound as if it is produced from an audio source location corresponding to that virtual surface. How the audio is provided may be determined based on the position of the viewer (e.g., the user or his/her device) relative to the presented visual content.

Visual content presentation with viewer position-based audio

An audio processing device may generate a plurality of microphone signals from a plurality of microphones of the audio processing device. The audio processing device may determine a gaze of a user who is wearing a playback device that is separate from the audio processing device, the gaze of the user being determined relative to the audio processing device. The audio processing device may extract speech that correlates to the gaze of the user, from the plurality of microphone signals of the audio processing device by applying the plurality of microphone signals of the audio processing device and the gaze of the user to a machine learning model. The extracted speech may be played to the user through the playback device.

Machine learning and user driven selective hearing

An audio renderer can have a machine learning model that jointly processes audio and visual information of an audiovisual recording. The audio renderer can generate output audio channels. Sounds captured in the audiovisual recording and present in the output audio channels are spatially mapped based on the joint processing of the audio and visual information by the machine learning model. Other aspects are described.

Audio renderer based on audiovisual information

A device with microphones can generate microphone signals during an audio recording. The device can store, in an electronic audio data file, the microphone signals, and metadata that includes impulse responses of the microphones. Other aspects are described and claimed.

Spatial audio file format for storing capture metadata

Each of a plurality of virtual loudspeaker arrays and their channels are produced, based on a corresponding microphone array and microphone signals thereof. Channels of a hallucinated loudspeaker array are determined based on the channels of the plurality of virtual loudspeaker arrays. The plurality of virtual loudspeaker arrays and the hallucinated loudspeaker array share a common geometry and orientation. Spatial audio is rendered based on the channels of the hallucinated loudspeaker array.

Auditory origin synthesis

Various implementations disclosed herein include devices, systems, and methods that that modify audio of played back AV content based on context in accordance with some implementations. In some implementations audio-visual content of a physical environment is obtained, and the audio-visual content includes visual content and audio content that includes a plurality of audio portions corresponding to the visual content. In some implementations, a context for presenting the audio-visual content is determined, and a temporal relationship between one or more audio portions of the plurality of audio portions and the visual content is determined based on the context. Then, synthesized audio-visual content is presented based on the temporal relationship.

Variable audio for audio-visual content

A machine learning model can determine output frequency response at different directions relative to a target audio output format, based on input including frequency response at directions relative to microphones of a capture device. A spatial filter determined based on the output frequency responses is applied to one or more of the microphone signals to map the spatial information from the microphone signals to the target audio output.

Deep-learning based beam forming synthesis for spatial audio

Spatial filters are generated that map response of an audio capture device to head related transfer functions (HRTFs) for different positions of the audio capture device relative to the HRTFs. A current set of spatial filters are determined based on the plurality of spatial filters and a head position of a user. The microphone signals are convolved with the current set of spatial filters, resulting in a left audio channel and right audio channel that form output binaural audio channels. The binaural audio channels can be used to drive speakers of a headphone set to generate sound that is perceived to have a spatial quality. Other aspects are described and claimed.

Growing community of inventors

Los Angeles, CA, United States of America

Symeon Delikaris Manias