Apple Inc.

Amazon Technologies, Inc.

Alexander Kanaris

Aram M Lindahl

Sorin V Dusan

Berkant Tacer

Gurhan Saplakoglu

Matthew Emanuel Last

Carlo Murgia

Henry Hung Yang

Esge Brandt Andersen

Karthik Jayaraman Raghuram

Richard Michael Powell

Joseph M Williams

Kuan-Chieh Yen

Vasu Iyengar

Wesley Scott Smith

David T Yeh

James Michael Silva

Barker N Bhaskaran

A system configured to perform source classification using a continuously active beamformer and a classifier to improve beamforming. A device processes audio data representing sounds from multiple sound sources to determine a source direction corresponding to each detected sound source. For each audio frame, the classifier receives a source direction for each unknown sound source and uses the source direction to determine first spectral characteristics that are unique to the individual sound source. By comparing the first spectral characteristics to spectral characteristics associated with labeled sound sources, the device identifies a match and associates the source directions with a corresponding labeled sound source. For each labeled sound source, the classifier determines attributes of a corresponding signal, and the device uses these attributes to select a single sound source. Using a desired look direction associated with the selected sound source, the beamformer generates audio data representing desired speech.

Sound source classification and beamforming

This disclosure describes, in part, techniques to process audio signals to lessen the impact that wind and/or other environmental noise has upon the resulting quality of audio signals output on speaker(s) of wireless earbuds or other headphones. For example, the techniques may determine a level of wind and/or other noise in an environment and may determine how best to process the signals to lessen the impact of the noise, such that one or more users that hear audio based on output of the signals hear higher-quality audio. In some instances, the techniques are able to detect levels of wind in a relatively low-power manner and as part of regular operation of the wireless earbuds or other headphones.

Detecting wind events in audio data

Maintaining time synchronization between clocks of different devices allows various operations, such as accurate timestamping input data, coordinating presentation of output, and so forth. A pairwise synchronization between a first and second device is provided by opportunistically including timing data within packets transmitted between the devices. A clock sync module in a second device uses a software-implemented phase-locked loop (PLL) with a software-implemented proportional-integral-derivative (PID) controller to synchronize a second clock in the second device to a first clock of the first device. The use of the clock sync module to process the timing data provides clock synchronization without introducing jitter or other undesirable effects at the second device. Using the clock sync module, the timing of the second clock of the second device quickly converges with the first clock. The close convergence facilitates time-sensitive operations, such as synchronizing presentation of content from many participating wireless devices.

System for time synchronization

A group of devices acquire audio input of a sound, such as speech, using respective microphones. For pairs of devices in the group, intensity of energy of audio input at each of the devices in the pair is used to determine first proximity data. Relative differences in time-of-arrival of the sound at the devices in the pair is used to determine second proximity data. The first and second proximity data are used to determine an estimated closest device of the pair with respect to the sound. Comparison of the first proximity data to the second proximity also allows a confidence value to be associated with the estimated closest device. The estimated closest device with the greatest confidence value may be selected for use to acquire audio input, present output, and so forth. Additional techniques such as beamforming techniques may be applied to the audio input from the selected device.

System to select audio from multiple connected devices

A system configured to perform distributed echo cancellation processing to attenuate feedback echo from occurring when two devices are acoustically coupled during a communication session. To reduce the feedback echo, one of the devices is configured as a hub device and receives microphone signals, synchronizes the microphone signals, and generates a mixed microphone signal. To enable distributed echo cancellation, the system includes bidirectional feedback link(s) between the hub device and each device synchronized with the hub device. For example, a first bidirectional feedback link sends a microphone signal from a second device to the hub device and sends the mixed microphone signal from the hub device to the second device, which the second device uses to perform echo cancellation. In addition, a second bidirectional feedback link sends a playback signal from the hub device to the second device and sends the output of echo cancellation back to the hub device.

Distributed feedback echo cancellation

Electronic system for audio noise processing and noise reduction comprises: first and second noise estimators, selector and attenuator. First noise estimator processes first audio signal from voice beamformer (VB) and generate first noise estimate. VB generates first audio signal by beamforming audio signals from first and second audio pick-up channels. Second noise estimator processes first and second audio signal from noise beamformer (NB), in parallel with first noise estimator and generates second noise estimate. NB generates second audio signal by beamforming audio signals from first and second audio pick-up channels. First and second audio signals include frequencies in first and second frequency regions. Selector's output noise estimate may be a) second noise estimate in the first frequency region, and b) first noise estimate in the second frequency region. Attenuator attenuates first audio signal in accordance with output noise estimate. Other embodiments are also described.

System and method for audio noise processing and noise reduction

A device can estimate the heart rate of an active user by using a physiological model to refine a 'direct' measurement of the user's heart rate obtained using a pulse sensor. The physiological model can be based on heart rate response to activity and can be informed by context information, such as the user's current activity and/or intensity level as well as user-specific parameters such as age, gender, general fitness level, previous heart rate measurements, etc. The physiological model can be used to predict a heart rate, and the prediction can be used to assess or improve the direct measurement.

Context-aware heart rate estimation

Embodiments of the invention determine whether speaker earbuds of a headset are positioned in a user's ears. The headset may be a 'Y' shaped headset with two earbuds having speakers and a plug for insertion into a jack of the audio device. Multiple microphones are located on wired lengths to the earbuds and a common wire between the lengths and the plug, to receive speech from the user's mouth. Each earbud may have a front and rear microphone, and an accelerometer. Embodiments can detect user speech vibrations at one or more of the microphones, and in the accelerometers in the earbuds. Based on these detections, it can be determined whether one or both of the earbuds are in user's ears. To provide more accurate beamforming, when only one of the earbuds is in the user's ears, only the microphones leading to that earbud are selected for beamforming input.

Detecting the positions of earbuds and use of these positions for selecting the optimum microphones in a headset

Digital signal processing microphone occlusion detection is described that can be used with a noise suppression system that uses two types of noise estimators, including a more aggressive one based on two audio signals (such as for non-stationary noises) and a less aggressive one based on one audio signal (such as for stationary noises). Decisions are made on how to select or combine the outputs of the noise estimators into a usable noise estimate, based on an occlusion function. The occlusion detection may alternatively be used to trigger an alert to users of multi-microphone audio processing systems, such as smart phones, headsets, laptops and tablet computers. Other embodiments are also described and claimed.

Microphone occlusion detector

A personal audio device has a bone conduction pickup transducer, having a housing of which a rigid outer wall has an opening formed therein. A volume of yielding material fills the opening in the rigid outer wall. An electronic vibration sensing element is embedded in the volume of yielding material. The housing is shaped, and the opening is located, so that the volume of yielding material comes into contact with an ear or cheek of a user who is using the personal audio device. Other embodiments are also described and claimed.

Bone-conduction pickup transducer for microphonic applications

Techniques are presented for dynamically controlling an audio crossfade from a first audio track to a second audio track. The first audio track may be decoded and buffered to a first buffer. A user may select a second audio track at some point during playback of the first audio track. The second audio track may be decoded, and a processor may access the decoded second audio track and the buffered first audio track and mix the two audio tracks. The mixed audio track may be inserted in the same first buffer near or at the read pointer of the first buffer, such that latency between the user selection and the crossfade into the selected audio track may be minimized. The insertion of the mixed audio track may depend on various factors, including the type of audio file and/or the decoder type used to decode the second audio track.

Growing community of inventors

San Jose, CA, United States of America

Alexander Kanaris