Sparse feature extraction using generalised partial least squares

We describe a general framework for feature extraction based on the deflation scheme used in partial least squares (PLS). The framework provides many desirable properties, such as conjugacy and efficient computation of the resulting features. When the projection vectors are constrained in a certain way, the resulting features have dual representations. Using the framework, we derive two new sparse feature extraction algorithms, sparse maximal covariance (SMC) and sparse maximal alignment (SMA). These algorithms produce features which are competitive with those extracted by kernel boosting, boosted latent features (BLF) and sparse kernel PLS on several UCI datasets. Furthermore, the sparse algorithms are shown to improve the performance of an SVM on a sample of the Reuters corpus volume 1 dataset.