Deep Feature Selection using a Teacher-Student Network

摘要: High-dimensional data in many machine learning applications leads to computational and analytical complexities. Feature selection provides an effective way for solving these problems by removing irrelevant redundant features, thus reducing model complexity improving accuracy generalization capability of the model. In this paper, we present a novel teacher-student feature (TSFS) method which 'teacher' (a deep neural network or complicated dimension reduction method) is first employed learn best representation low dimension. Then 'student' simple network) used perform minimizing reconstruction error dimensional representation. Although scheme not new, our knowledge, it time that selection. The proposed TSFS can be both supervised unsupervised This evaluated on different datasets compared with state-of-the-art existing methods. results show performs better terms classification clustering accuracies error. Moreover, experimental evaluations demonstrate degree sensitivity parameter method.