Multi-branched network for event detection

Abstract

A system that performs event detection using a multi-branched network for sensor fusion. For example, a device may detect when a tap event occurs on a surface of the device using a combination of microphone audio data and sensor data, such as motion data generated by a motion sensor. Prior to combining these inputs for further inference, the device may use separate neural networks to independently extract features from the audio data and the sensor data. This improves an accuracy of tap detection and enables detection of additional tap gestures and/or other types of event/activity detection, such as typing detection. The multi-branched network may generate fused data by processing audio features, motion data, raw audio data, raw accelerometer data, and/or additional sensor data. Depending on the inputs, a number of branches, a branch depth, and/or a number of event detectors may vary without departing from the disclosure.