Lemon Inc.

Bytedance Inc.

David Trevelyan

Jitong Chen

Edmund Newton-Rex

Jason Storey

Matthew David Avent

Pierre Chanquion

Lamtharn Hantrakul

Tao Xiong

Maryyann Crichton

Nathanael Schager

Bochen Li

Fraser Smith

Josiah John Serrano

Suiyu Feng

Jonathan Cooper

Chenyu Sun

Hwankyoo Shawn Kim

Xuefan Hu

Brandon Wu

Siyuan Shan

Haonan Chen

Eamon Hyland

Janne Jayne Harm Renée Spijkervet

Example aspects include techniques for implementing real-time and low-latency synthesis of audio. These techniques may include generating a frame by sampling audio input in increments equal to a buffer size of until a threshold corresponding to a frame size used to train a machine learning (ML) model is reached, detecting feature information within the frame, determining, by the ML model, control information for audio reproduction based on the feature information. In addition, the techniques may include generating filtered noise information by inverting the noise magnitude control information using an overlap and add technique, generating, based on the control information, additive harmonic information by combining a plurality of scaled wavetables, and rendering audio output based on the filtered noise information and the additive harmonic information.

Method and system for real-time and low latency synthesis of audio using neural networks and differentiable digital signal processors

The present disclosure describes techniques for differentiable wavetable synthesizer. The techniques comprise extracting features from a dataset of sounds, wherein the features comprise at least timbre embedding; input the features to the first machine learning model, wherein the first machine learning model is configured to extract a set of N×L learnable parameters, N represents a number of wavetables, and L represents a wavetable length; outputting a plurality of wavetables, wherein each of plurality of wavetables comprises a waveform associated with a unique timbre, the plurality of wavetables form a dictionary, and the plurality of wavetables are portable to perform audio-related tasks. Finally, the said wavetables are used to initialize another machine learning model so as to help reduce computational complexity of an audio synthesis obtained as output of the another machine learning model.

Differentiable wavetable synthesizer using plurality of machine learning models to reduce computational complexity of audio synthesis

The present application provides a special effect processing method and apparatus. The method includes: generating an audio signal in response to a touch operation of a user in a process of playing a video; segmenting the audio signal into multiple audio frames; performing, according to attributes of the audio frames, special effect processing on a picture which is currently played in the video.

Special effect processing method and apparatus

A music production system comprises: a computer interface comprising at least one input for receiving an external request for a piece of music and at least one output for transmitting a response to the external request which comprises or indicates a piece of music incorporating first music data; a first music production component configured to process second music data according to at least a first input setting so as to generate the first music data; a second music production component configured to receive via the computer interface an internal request, and provide the second music data based on at least a second input setting denoted by the internal request; and a controller configured to determine in response to the external request the first and second input settings, and instigate the internal request via the computer interface.

Modular automated music production server

A method of combining data, the method comprising: receiving video data, the video data corresponding to recorded video having a video duration determined by a user; selecting backing audio data, the backing audio data corresponding to backing audio having a predetermined duration; determining a difference between the predetermined duration and the video duration; and modifying the backing audio data by adjusting the predetermined duration based on the video duration to create an adjusted predetermined duration, the adjusted predetermined duration being such that the backing audio and recorded video may be simultaneously output in synchronisation.

Method of combining data

Growing community of inventors

London, United Kingdom

David Trevelyan