Rensselaer Polytechnic Institute

Jonathan Mathews

Jonas Braasch

Richard J Radke

Devavrat Jivani

Gyanendra Sharma

A time-of-flight (TOF) array including a plurality of cameras is positioned with a coplanar Higher Order Ambisonics (HOA) array including a plurality of microphones above a target area such as a room or space for analyzing speaker activity in that area. The HOA microphone array iteratively samples sound energy level frames captured by the microphones to identify a sound vector associated with the global maximum energy level in the sound energy level frame. This sound vector is fused with the data from the TOF camera array that identified the positions of sound sources in the target area to associate produced sound corresponding with the sound vector with a physical active sound source in the target area. A beamformer can then be used to save audio corresponding to the sound vector and discard audio not associated with the sound vector to produce a sound recording associated with the active sound source.

Growing community of inventors

Troy, NY, United States of America

Jonathan Mathews