Microsoft Technology Licensing, LLC

Meta Platforms Technologies, LLC

Facebook Technologies, LLC

Andrew Lovitt

Philip Robinson

Antonio John Miller

Michael Anthony Hall

Jason Ryan Farmer

Scott Selfon

William Owen Brimijoin, Ii

Carl Schissler

Peter Henry Maresh

Scott Phillip Selfon

Morteza Khaleghimeybodi

Jacob Ryan Donley

Sebastià Vicenç Amengual Gari

Crystal Lee Parker

Mark Louis Wilson O'Hanlon

Darko Kirovski

Scott Phillips Porter

Vladimir Tourbabin

Jeremiah Spradlin

Ted E Dinklocker

Peter Harty Dodds

Craig Scott Brenner

Stefanie Lyn Tomko

Sebastiá Vicenç Amengual Gari

Taher Shahbazi Mirzahasanloo

Yun-Cheng Ju

Sebastian Sztuk

Tony David

Richard Dennis Beckert

Nicholas William Metzar

Tetsuro Oishi

Nils Thomas Fritiof Lunner

Nava K Balsam

Steven P Maillet

Gordon George Free

Kenneth Harry Cooper

Samuel Steele Noertker

Syavosh Zadissa

Nathan M Rau

Wei-Ting Frank Liu

Sebastiá Vicenç Amengual Garí

Igor D Spivak

Phillip G Hays

Sean Allyn Coffin

Sebastiá Vicenc̨ Amengual Gari

William Owen Ii Brimijoin

The disclosed computer-implemented method may include applying, via a sound reproduction system, sound cancellation that reduces an amplitude of various sound signals. The method further includes identifying, among the sound signals, an external sound whose amplitude is to be reduced by the sound cancellation. The method then includes analyzing the identified external sound to determine whether the identified external sound is to be made audible to a user and, upon determining that the external sound is to be made audible to the user, the method includes modifying the sound cancellation so that the identified external sound is made audible to the user. Various other methods, systems, and computer-readable media are also disclosed.

Adaptive ANC based on environmental triggers

An audio system receives sounds from a local area with a set of sensors. The audio system analyzes the sounds to identify an amount of background noise in the environment. The audio system may also identify target sound in the environment, such as voice from a nearby speaker, to reinforce. The amount of background noise is then used to adaptively set a level of reinforcement for the target sound. In low-noise environments, the target sound may be minimally, or not reinforced, while in high-noise environments a reinforcement sound signal may be generated with the target sound so that a transducer array may generate reinforcing sounds for the target sound and increase its contrast with the noise in the environment. By applying the reinforcement dynamically, the system automatically adjusts for different sounds and environments.

Augmented hearing via adaptive self-reinforcement

A method to assess user condition for wearable devices using electromagnetic sensors is provided. The method includes receiving a signal from an electromagnetic sensor, the signal being indicative of a health condition of a user of a wearable device, selecting a salient attribute from the signal, and determining, based on the salient attribute, the health condition of the user of the wearable device. A non-transitory, computer-readable medium storing instructions which, when executed by a processor, cause a system to perform the above method, and the system, are also provided.

One-touch spatial experience with filters for AR/VR applications

A communication system actively monitors a conversation between users in an environment. Once a threshold condition occurs (e.g., client device within a threshold distance of a room where the conversation is occurring, the user enters a chatroom via the client device), the client device receives a summary from the communication system. The communication system may customize the summary in accordance with user preferences. User preferences may include level of detail in summary or time range associated with summary. The client device then presents the summary to the user. The summary may be audio or text. In some embodiments, the summary may be historical audio that is overlaid over current audio at faster speed and higher volume, and stops once the historical audio reaches real time.

Generating a summary of a conversation between users for an additional user in response to determining the additional user is joining the conversation

An audio system determines latency for each client device of users participating in an artificial reality or augmented reality environment. The latency determined for a client device accounts for latency of a communication channel used by the client device to exchange data, as well as time for the client device to perform computational tasks. The audio system determines a distance in the artificial reality environment between a user's avatar and an audio source for audio data and adjusts one or more transfer functions for generating audio content for the user to account for effects of the latency determined for the user's client device on the audio content. In some embodiments, the audio system modifies a distance in the artificial reality environment between the user's avatar and the audio source to compensate for latency of the user's client device.

Adjusting generation of spatial audio for a receiving device to compensate for latency in a communication channel between the receiving device and a sending device

An electronic system is described that is responsive to non-verbal commands. The system may include a display component for presenting visual content to a user and an audio system, such as an in-ear or over-the-ear speaker system. The speaker system may be equipped with a bone conduction sensor or in-ear microphone that generates data associated with vibrations of a facial region of the user. The system may use the data to detect non-verbal command in the form of grunts, hums, coughs, tongue clicks, and the like. The system may perform various predetermined operations based on the detected vibrations.

System for non-verbal hands-free user input

A method to combine contact and acoustic microphones in a headset for voice wake and voice processing in immersive reality applications is provided. The method includes receiving, from a contact microphone, a first acoustic signal, determining a fidelity and a quality of the first acoustic signal, receiving, from an acoustic microphone, a second acoustic signal, and when the fidelity and quality of the first acoustic signal exceeds a pre-selected threshold, combining the first acoustic signal and the second acoustic signal to provide an enhanced acoustic signal to a smart glass user. A non-transitory, computer-readable medium storing instructions to cause a headset to perform the above method, and the headset, are also provided.

Contact and acoustic microphones for voice wake and voice processing for AR/VR applications

A communication system provides audio content to one or more client devices capable of playing spatialized audio content. For example, the communication system receives audio content from a client device and transmits the audio content to other client devices to be played for users. The communication system dynamically modifies audio content transmitted to different client devices based on a payload including audio parameters (e.g., local area acoustic properties, an audiogram for a user, a head related transfer function for a user, etc.) received from a client device.

Modifying audio data transmitted to a receiving device to account for acoustic parameters of a user of the receiving device

A method of tracking sound sources in an environment by monitoring the local area with a primary sensor system comprising a microphone array. The location of sound sources in the local area are tracked using the monitored sound. If an ambiguity is determined in the tracked location of the sound source, a secondary sensor system is activated. The secondary sensor system has a larger power draw than the primary sensor system. The secondary sensor system determines the updated location of the sound source. The tracked location of the sound sources is updated to be the updated location. Once the location is updated, the secondary sensor system is deactivated.

Power efficient acoustic tracking of sound sources

A method for audio beam steering, tracking, and audio effects for an immersive reality application is provided. The method includes receiving, from an immersive reality application, a first audio waveform from a first acoustic source to provide to a user of a headset, identifying a perceived direction for the first acoustic source relative to the headset based on a location of the first acoustic source, and providing, to a first speaker in a client device, an audio signal including the first audio waveform, wherein the audio signal includes a time delay and an amplitude of the first audio waveform based on the perceived direction. A non-transitory, computer-readable medium storing instructions which, when executed by a processor, cause a system to perform the above method, and the system, are also provided.

Audio beam steering, tracking and audio effects for AR/VR applications

An audio system presents enhanced audio content to a user of a headset. The audio system detects sounds from the local area, at least a portion of which originate from a human sound source. The audio system obtains a voice profile of an identifies human sound source that generates at least the portion of the detected sounds. Based in part on the voice profile, the audio system enhances the portion of the detected sounds that are generated by the human sound source to obtain enhanced audio. The audio system presents the enhanced audio to the user.

Social network based voice enhancement system

Validation of an activation event of a device based on analyzing audio data is disclosed. The device detects an activation event, e.g., an input to a soft-touch button or a wake word detected by a microphone array. Responsive to detecting the activation event, the device captures, via a microphone array on the device, sound from a local area of the device. The device stores the captured sound as audio data in an audio buffer. The device performs a validation of the activation event by analyzing the captured sound. The device performs an action based on a result of the validation.

Headset activation validation based on audio data

This disclosure describes embodiments of methods, non-transitory computer-readable media, and systems for detecting that a physical space includes a physical object corresponding to an analogous virtual object from an augmented reality experience and rendering (or otherwise modifying) the augment reality experience to integrate the physical object as part of the experience. In particular, the disclosed systems can determine that a physical object within a physical environment corresponds to an analogous virtual object of an augmented reality experience. Based on this correspondence, the disclosed systems can modify one or more of the virtual graphics, sound, or other features corresponding to the augmented reality experience to represent the virtual object using the physical object. For example, the disclosed systems can modify acoustic features of a sound for the augmented reality experience to simulate the sound originating from (or being affected by) the physical object.

Generating augmented reality experiences utilizing physical objects to represent analogous virtual objects

A method comprising determining a set of position parameters for an inertial measurement unit (IMU) on a headset worn by a user. The set of position parameters includes at least a first yaw measurement and a first roll measurement. The set describes a pointing vector. The method further comprises calculating a drift correction component that describes a rate of correction. The drift correction component is based at least in part on the set of position parameters. The method further comprises applying the drift correction component to one or more subsequent yaw measurements for the IMU. The drift correction component forces an estimated nominal position vector to the pointing vector at the rate of correction.

Biologically-constrained drift correction of an inertial measurement unit

Growing community of inventors

Redmond, WA, United States of America

Andrew Lovitt