Apple Inc.

Other

Joshua D Atkins

Mehrez Souden

Jason Wung

Jonathan D Sheaffer

Ante Jukic

Symeon Delikaris Manias

Ramin Pishehvar

Martin E Johnson

Ismael Hamad Nawfal

Guy C Nicholson

Stephen J Nimick

Jason M Harlow

Christopher T Eubank

Juha O Merimaa

Ashrith Deshpande

Soenke Pelzer

Dirk Schroeder

Peter A Raffensperger

Darius A Satongar

Stuart John Wood

Sean Anthony Ramprashad

Tomlinson M Holman

Vasu Iyengar

Daniele Giacobello

Tarun Pruthi

Lance F Reichert

Feipeng Li

Aram M Lindahl

Dipanjan Sen

Sawyer Isaac Cohen

Frank M Baumgarte

Esge Brandt Andersen

Justin Derry Crosby

Sorin V Dusan

John Woodruff

Eric A Allamanche

Yoon Kim

Lalin S Theverapperuma

Sarmad Aziz Malik

Adam Strauss

Raghavendra S Prabhu

John Bridle

Gaetan R Lorho

Symeon Delikaris-Manias

Feiping Li

Ramin Pichevar

Abdullah Fahim

Andrey D Del Pozo

An audio device can sense sound in a physical environment using a plurality of microphones to generate a plurality of microphone signals. Clean speech can be extracted from microphone signals. Ambience can be extracted from the microphone signals. The clean speech can be encoded at a first compression level. The ambience can be encoded at a second compression level that is higher than the first compression level. Other aspects are also described and claimed.

Audio encoding with compressed ambience

Processing sound in an enhanced reality environment can include generating, based on an image of a physical environment, an acoustic model of the physical environment. Audio signals captured by a microphone array, can capture a sound in the physical environment. Based on these audio signals, one or more measured acoustic parameters of the physical environment can be generated. A target audio signal can be processed using the model of the physical environment and the measured acoustic parameters, resulting in a plurality of output audio channels having a virtual sound source with a virtual location. The output audio channels can be used to drive a plurality of speakers. Other aspects are also described and claimed.

Processing sound in an enhanced reality environment

Processing of ambience and speech can include extracting from audio signals, ambience and speech signals. One or more spatial parameters can be generated that define spatial characteristics of ambience sound in the one or more ambience audio signals. The primary speech signal, the one or more ambience audio signals, and the spatial parameters can be encoded into one or more encoded data streams. Other aspects are described and claimed.

Separating and rendering voice and ambience signals by offsetting impact of device movements

Digital audio signal processing techniques used to provide an acoustic transparency function in a pair of headphones. A number of transparency filters can be computed at once, using optimization techniques or using a closed form solution, which are based on multiple re-seatings of the headphones and that are as a result robust for a population of wearers. In another embodiment, a transparency hearing filter of a headphone is computed by an adaptive system that takes into consideration the changing acoustic to electrical path between an earpiece speaker and an interior microphone of that headphone while worn by a user. Other embodiments are also described and claimed.

Spatial headphone transparency

An audio processing device may generate a plurality of microphone signals from a plurality of microphones of the audio processing device. The audio processing device may determine a gaze of a user who is wearing a playback device that is separate from the audio processing device, the gaze of the user being determined relative to the audio processing device. The audio processing device may extract speech that correlates to the gaze of the user, from the plurality of microphone signals of the audio processing device by applying the plurality of microphone signals of the audio processing device and the gaze of the user to a machine learning model. The extracted speech may be played to the user through the playback device.

Machine learning and user driven selective hearing

Disclosed is a multi-task machine learning model such as a time-domain deep neural network (DNN) that jointly generate an enhanced target speech signal and target audio parameters from a mixed signal of target speech and interference signal. The DNN may encode the mixed signal, determine masks used to jointly estimate the target signal and the target audio parameters based on the encoded mixed signal, apply the mask to separate the target speech from the interference signal to jointly estimate the target signal and the target audio parameters, and decode the masked features to enhance the target speech signal and to estimate the target audio parameters. The target audio parameters may include a voice activity detection (VAD) flag of the target speech. The DNN may leverage multi-channel audio signal and multi-modal signals such as video signals of the target speaker to improve the robustness of the enhanced target speech signal.

End-to-end time-domain multitask learning for ML-based speech enhancement

A method performed by a programmed processor of an audio system, the method includes receiving a sound track that has a track length, producing a binaural audio version of a sound track, the binaural audio version having an extended track length performing a fading operation upon the binaural audio version to gradually reduce a signal level of the binaural audio version to below a signal threshold level at a time along the extended track length that corresponds to an end time of the track length of the sound track; and storing the binaural audio version having the track length of the sound track in memory for later transmission to an audio playback device for driving one or more speakers.

Method and system for maintaining track length for pre-rendered spatial audio

A plurality of microphone signals can be captured with a plurality of microphones of the device. One or more echo dominant audio signals can be determined based on a pick-up beam directed towards one or more speakers of a playback device. Sound that is emitted from the one or more speakers and sensed by the plurality of microphones can be removed from plurality of microphone signals, by using the one or more echo dominant audio signals as a reference, resulting in clean audio.

Spatially informed acoustic echo cancelation

A device with microphones can generate microphone signals during an audio recording. The device can store, in an electronic audio data file, the microphone signals, and metadata that includes impulse responses of the microphones. Other aspects are described and claimed.

Spatial audio file format for storing capture metadata

Digital audio signal processing techniques used to provide an acoustic transparency function in a pair of headphones. A number of transparency filters can be computed at once, using optimization techniques or using a closed form solution, that are based on multiple re-seatings of the headphones and that are as a result robust for a population of wearers. In another embodiment, a transparency hearing filter of a headphone is computed by an adaptive system that takes into consideration the changing acoustic to electrical path between an earpiece speaker and an interior microphone of that headphone while worn by a user. Other embodiments are also described and claimed.

Systems and processes for operating an intelligent automated assistant are provided. In accordance with one example, a method includes, at an electronic device with one or more processors, memory, and a plurality of microphones, sampling, at each of the plurality of microphones of the electronic device, an audio signal to obtain a plurality of audio signals; processing the plurality of audio signals to obtain a plurality of audio streams; and determining, based on the plurality of audio streams, whether any of the plurality of audio signals corresponds to a spoken trigger. The method further includes, in accordance with a determination that the plurality of audio signals corresponds to the spoken trigger, initiating a session of the digital assistant; and in accordance with a determination that the plurality of audio signals does not correspond to the spoken trigger, foregoing initiating a session of the digital assistant.

Detecting a trigger of a digital assistant

A device implementing a system for processing speech in an audio signal includes at least one processor configured to receive an audio signal corresponding to at least one microphone of a device, and to determine, using a first model, a first probability that a speech source is present in the audio signal. The at least one processor is further configured to determine, using a second model, a second probability that an estimated location of a source of the audio signal corresponds to an expected position of a user of the device, and to determine a likelihood that the audio signal corresponds to the user of the device based on the first and second probabilities.

Spatially informed audio signal processing for user speech

A device for processing audio signals in a time-domain includes a processor configured to receive multiple audio signals corresponding to respective microphones of at least two or more microphones of the device, at least one of the multiple audio signals comprising speech of a user of the device. The processor is configured to provide the multiple audio signals to a machine learning model, the machine learning model having been trained based at least in part on an expected position of the user of the device and expected positions of the respective microphones on the device. The processor is configured to provide an audio signal that is enhanced with respect to the speech of the user relative to the multiple audio signals, wherein the audio signal is a waveform output from the machine learning model.

Microphone array based deep learning for time-domain speech signal extraction

Growing community of inventors

Los Angeles, CA, United States of America

Joshua D Atkins