Audio Style Transfer
Position : Research Assitant
Workplace : AI Media Lab, Art&Technology Department, Sogang University
Category : media technology research, audio generation, deep learning, virtual machines, cloud services
Timeline : Undefined
Skills : software testing, programming, sound generation
Concept
I remembered an old project from a few years ago that I left in lack of optimization: Neural Style Transfer.
The idea was to transfer one musical style onto the content of a piece of music. For example, take a piece of classical music and change it into jazz.
How it works
Style Transfer techniques were efficient on images when I first started doing this research, and heavy models would work for music. Those models would take time to train and require a lot of data and memory space, so my research led me to Ulyanov’s paper, made of a lighter and faster model.
I experimented with different inputs: waveform, Short Fourier Transform and Constant Q transform. The code is also setup so layers can be removed or added from the neural network.
The goal is to dive back into an old project, upgrade and optimize as I explain my way through it. It should be updated.
Tools I used
-
Python : Pytorch, librosa,
-
Google Colab, Jupyter notebooks
-
Github, Git