Supervised low rank matrix approximation for stable feature selection

Increasing attention has been focused on the stability of selected features or selection stability, which is becoming a new measure in determining the effectiveness of a feature selection algorithm besides the learning performance. A recent study has shown that data characteristics play a significant role in selection stability. Hence, the solution to selection instability should begin with data. In this work, we propose a novel framework with a noise-reduction step before feature selection. Noise reduction is achieved via well-known low rank matrix approximation techniques (namely SVD and NMF) in a supervised manner to reduce data noise and variance between samples from the same class. The new framework is empirically shown to be highly effective with real high-dimensional datasets improving both selection stability and the precision of selecting relevant features while maintaining the classification accuracy for various feature selection methods.