The paper discussed at this journal club focused on using GANs to generate music, specifically samples with a particular pitch and timbre. Different forms of input data are considered and compared, including time series and different spectrograms. Results are compared to other approaches such as WaveGAN and WaveNet.

Abstract

Efficient audio synthesis is an inherently difficult machine learning task, as human perception is sensitive to both global structure and fine-scale waveform coherence. Autoregressive models, such as WaveNet, model local structure at the expense of global latent structure and slow iterative sampling, while Generative Adversarial Networks (GANs), have global latent conditioning and efficient parallel sampling, but struggle to generate locally-coherent audio waveforms. Herein, we demonstrate that GANs can, in fact, generate high-fidelity and locally-coherent audio by modelling log magnitudes and instantaneous frequencies with sufficient frequency resolution in the spectral domain. Through extensive empirical investigations on the NSynth dataset, we demonstrate that GANs are able to outperform strong WaveNet baselines on automated and human evaluation metrics, and efficiently generate audio several orders of magnitude faster than their autoregressive counterparts.

The paper is available here. The code is available in a github repository and there is also a google colab notebook and suplmentary page with samples.