Lemon Inc.

Lamtharn Hantrakul

David Trevelyan

Matthew David Avent

Jitong Chen

Janne Jayne Harm Renée Spijkervet

Haonan Chen

Siyuan Shan

Example aspects include techniques for implementing real-time and low-latency synthesis of audio. These techniques may include generating a frame by sampling audio input in increments equal to a buffer size of until a threshold corresponding to a frame size used to train a machine learning (ML) model is reached, detecting feature information within the frame, determining, by the ML model, control information for audio reproduction based on the feature information. In addition, the techniques may include generating filtered noise information by inverting the noise magnitude control information using an overlap and add technique, generating, based on the control information, additive harmonic information by combining a plurality of scaled wavetables, and rendering audio output based on the filtered noise information and the additive harmonic information.

Method and system for real-time and low latency synthesis of audio using neural networks and differentiable digital signal processors

The present disclosure describes techniques for differentiable wavetable synthesizer. The techniques comprise extracting features from a dataset of sounds, wherein the features comprise at least timbre embedding; input the features to the first machine learning model, wherein the first machine learning model is configured to extract a set of N×L learnable parameters, N represents a number of wavetables, and L represents a wavetable length; outputting a plurality of wavetables, wherein each of plurality of wavetables comprises a waveform associated with a unique timbre, the plurality of wavetables form a dictionary, and the plurality of wavetables are portable to perform audio-related tasks. Finally, the said wavetables are used to initialize another machine learning model so as to help reduce computational complexity of an audio synthesis obtained as output of the another machine learning model.

Differentiable wavetable synthesizer using plurality of machine learning models to reduce computational complexity of audio synthesis

Growing community of inventors

Singapore, Singapore

Lamtharn Hantrakul