Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Jacob Ryan Donley

Vladimir Tourbabin

Antonio John Miller

Scott Phillips Porter

Ravish Mehra

Andrew Lovitt

Philip Robinson

Paul Thomas Calamia

Buye Xu

Nava K Balsam

Morteza Khaleghimeybodi

Ashley N Saulsbury

Pablo Francisco Faundez Hoffmann

Vamsi Krishna Ithapu

Nils Thomas Fritiof Lunner

Peter Harty Dodds

Cesare Valerio Parise

Sebastià Vicenç Amengual Gari

Vinay Kumar Kothapally

A device and/or method for storing one or more sound filters in a discretized pose space. The device determines that a microphone array during a first time period is in a first discrete pose of a plurality of discrete poses, wherein the plurality of discrete poses discretizes a pose space. The pose space includes at least an orientation component and may further include a translation component. The device retrieves a sound filter associated with the first discrete pose from a memory cache (e.g., for memoization). The device generates audio content using the sound filter and presents the audio content via a transducer array.

Memory recall of sound filter stored in association with a discrete pose for audio beamforming

An artificial reality headset enhances audio signals from a target sound source using information from other devices in the local area. A primary headset broadcasts a location of a target sound source to secondary headsets in a local area. The secondary headsets transmit audio signals to the primary headset to enhance the audio content presented by the primary headset to a user. The secondary headset may select an array transfer function for the location of the target sound source. The secondary headsets correlate known transfer functions in the target direction with estimated transfer functions. The secondary headset may perform beamforming on the target sound source and transmit the output audio signal to the primary headset. In some embodiments, the secondary headset may transmit the array transfer function and a raw audio signal to the primary headset. The primary headset generates audio content based on the received audio signal.

Distributed speech enhancement using generalized eigenvalue decomposition

A communication system provides audio content to one or more client devices capable of playing spatialized audio content. For example, the communication system receives audio content from a client device and transmits the audio content to other client devices to be played for users. The communication system dynamically modifies audio content transmitted to different client devices based on a payload including audio parameters (e.g., local area acoustic properties, an audiogram for a user, a head related transfer function for a user, etc.) received from a client device.

Modifying audio data transmitted to a receiving device to account for acoustic parameters of a user of the receiving device

An audio system presents enhanced audio content to a user of a headset. The audio system detects sounds from the local area, at least a portion of which originate from a human sound source. The audio system obtains a voice profile of an identifies human sound source that generates at least the portion of the detected sounds. Based in part on the voice profile, the audio system enhances the portion of the detected sounds that are generated by the human sound source to obtain enhanced audio. The audio system presents the enhanced audio to the user.

Social network based voice enhancement system

An electronic device localizes an audio source by normalizing an amplitude of an audio signal over a time period. The electronic device receives, from one or more microphones of the electronic device, signal(s) representative of audio emitted by an audio source over a time period. The electronic device estimates amplitudes of the signal(s) at a first time within the time period and at a second time within the time period, where the second time is different from the first time. The electronic device normalizes the amplitudes associated with the first and second times to generate normalized amplitudes. The electronic device determines a combined amplitude representative of the audio emitted by the audio source by combining the normalized amplitudes. The electronic device determines, based at least in part on the combined amplitude and motion of the electronic device, an estimated position of the audio source relative to the electronic device.

Audio source localization

A system and method for storing data samples in discrete poses and recalling the stored data samples for updating a sound filter. The system determines that a microphone array at a first time period is in a first discrete pose of a plurality of discrete poses, wherein the plurality of discrete poses discretizes a pose space. The pose space includes at least an orientation component and may further include a translation component. The system retrieves one or more historical data samples associated with the first discrete pose, generated from sound captured by the microphone array before the first time period, and stored in a memory cache (e.g., for memorization). The system updates a sound filter for the first discrete pose using the retrieved one or more historical data samples. The system generates and presents audio content using the updated sound filter.

Memory recall of historical data samples bucketed in discrete poses for audio beamforming

Determination of a composite acoustic parameter value for a headset is presented herein. A directionally enhanced audio signal is generated based on audio signals from an acoustic sensor array and a spatial signal enhancement filter that is directed for enhancement of a sound source. A SNR improvement value is determined based on a SNR value of the directionally enhanced audio signal and a SNR value of an audio signal from an acoustic sensor of the acoustic sensor array. The SNR improvement value is input into a model that maps SNR improvement values to spatial acoustic parameters to determine a spatial acoustic parameter. A temporal acoustic parameter is determined based on the audio signals. The composite acoustic parameter value is determined based on the spatial acoustic parameter and a temporal acoustic parameter value. Audio content presented to a user is adjusted based in part on the composite acoustic parameter value.

Determination of composite acoustic parameter value for presentation of audio content

The disclosed computer-implemented method may include receiving a signal for each channel of an audio transducer array on a wearable device. The method may also include calculating a beamformed signal for each beam direction of a set of beamforming filters for the wearable device. Additionally, the method may include classifying a first beamformed signal from the calculated beamformed signals into a first class of sound and a second beamformed signal from the calculated beamformed signals into a second class of sound. The method may also include adjusting, based on the classifying, a gain of the first beamformed signal relative to the second beamformed signal. Furthermore, the method may include converting the beamformed signals into spatialized binaural audio based on a position of a user. Finally, the method may include transmitting the spatialized binaural audio to a playback device. Various other methods, systems, and computer-readable media are also disclosed.

Systems and methods for classifying beamformed signals for binaural audio playback

An audio system for customizing sound fields for increased user privacy. A microphone array of a headset detects sounds from one or more sound sources in a local area of the headset. The audio system estimates array transfer functions (ATFs) associated with the sounds, and determines determining sound field reproduction filters for a loudspeaker array of the headset using the ATFs. The audio system presents audio content, via the loudspeaker array, based in part on the sound field reproduction filters. The presented audio content has a sound field that has a reduced amplitude in a first damped region of the local area that includes a first sound source of the one or more sound sources.

Customized sound field for increased privacy

A wearable device includes an audio system. In one embodiment, the audio system includes a sensor array that includes a plurality of acoustic sensors. When a user wears the wearable device, the audio system determines an acoustic transfer function for the user based upon detected sounds within a local area surrounding the sensor array. Because the acoustic transfer function is based upon the size, shape, and density of the user's body (e.g., the user's head), different acoustic transfer functions will be determined for different users. The determined acoustic transfer functions are compared with stored acoustic transfer functions of known users in order to authenticate the user of the wearable device.

Wearer identification based on personalized acoustic transfer functions

An audio system for a wearable device dynamically updates acoustic transfer functions. The audio system is configured to estimate a direction of arrival (DoA) of each sound source detected by a microphone array relative to a position of the wearable device within a local area. The audio system may track the movement of each sound source. The audio system may form a beam in the direction of each sound source. The audio system may identify and classify each sound source based on the sound source properties. Based on the DoA estimates, the movement tracking, and the beamforming, the audio system generates or updates the acoustic transfer functions for the sound sources.

Acoustic transfer function personalization using sound scene analysis and beamforming

The disclosed computer-implemented method may include accessing environment information identifying an undesired sound source within an environment. The method may further include determining, based on the accessed environment information, a spatial location of the undesired sound source in the environment that is to be attenuated using an active noise cancelling (ANC) signal. The method may also include forming a microphone beam, using two or more microphones, directed at the determined spatial location, where the microphone beam is configured to capture audio information from the determined spatial location. Still further, the method may include generating an ANC signal using the audio information captured using the microphone beam, and playing back the generated ANC signal to attenuate the undesired sound source at the determined spatial location. Various other methods, systems, and computer-readable media are also disclosed.

Selective active noise cancellation

An audio-video correspondence system performs a correspondence analysis between audio and video content. The system obtains video content that includes audio content, the video content comprising a plurality of frames. The system identifies a first set of pixels within the video content associated with changes in pixel values over a first set of frames of the video content. The system subsequently identifies, from the audio content, first audio associated with the first set of frames corresponding to changes in pixel values of the first set of pixels. The system finally assigns the first audio over the first set of frames to the first set of pixels.

Growing community of inventors

Kirkland, WA, United States of America

Jacob Ryan Donley