Channel-wise Subband Input for Better Voice and Accompaniment Separation on High Resolution Music

Table of Contents

Abstract

Link to our paper

Link to our code

Demos - The best performance we get with larger training set

Model:

MDN_K=4 (0.27 million parameters)

Unet-5_K=4 (13.3 million parameters)

Demos - The experiments in our paper

Genre:

Pop, Rap, Rock, Electronic, Folk, Others

Abstract

This paper presents a new input format, channel-wise subband input (CWS), for convolutional neural network (CNN) based music source separation (MSS) models in the frequency domain. We aim to address the major issues in CNN-based high-resolution MSS model: high computational cost and weight sharing between distinctly different bands. Specifically in this paper, we decompose the input mixture spectra into several bands and concatenate them channel-wise as the model input. The proposed approach enables effective weight sharing in each subband and introduces more flexibility between channels. For comparison purposes, we perform voice and accompaniment separation (VAS) on models with different scales, architectures, and CWS settings. The result shows that the CWS input is beneficial in many aspects. Among all our experiments, it enables models to obtain a 6.9% performance gain on average. With even a smaller number of parameters, much smaller training data, and shorter training time, our MDenseNet with 8-bands CWS input still surpasses the original MMDenseNet with a large margin. CWS also reduces computational cost and training time to a large extent, which can considerably expedite the experiment process.

Code

We open-source our code on github!