An attention-based transformer feature selection method for high-dimensional, low-sample gene expression data
Abstract
Gene expression data are usually characterized by high dimensionality and small sample size, posing significant challenges for feature selection and biomarker discovery in cancer classification. To address this problem, this study proposes a Transformer-based feature selection framework, termed Transformer Encoder Network (TEN). TEN designs a data augmentation strategy for high-dimensional, low-sample feature selection oriented to gene expression data. By expanding the training samples and combining the biological characteristics of gene expression data to enhance the differences among features, while utilizing the self-attention mechanism to model relational information among latent sample representations, TEN thereby facilitates the learning of class-discriminative structure. We conduct a systematic evaluation of TEN on ten benchmark datasets and further verify its effectiveness on five real gene-expression datasets (GSE68465, GSE12452, GSE76427, GSE14520, and GSE39582). All experiments are evaluated using a Support Vector Machine classifier (SVM), with TEN achieving classification Area Under the Curve (AUC) values of 1.0000, 0.9746, 0.9813, 0.9841, and 0.9960 on the above five datasets, respectively. The results show that TEN achieves favorable performance compared with multiple existing methods in the evaluated feature selection tasks, and the proposed data augmentation strategy helps to improve model performance under high-dimensional low-sample conditions. Furthermore, several genes selected by TEN are supported by differential expression analysis, survival analysis, and published literature. These genes are closely related to tumor development and patient prognosis, indicating that the features identified by this method not only have statistical advantages but also possess good biological interpretability. Clinical trial number: Not applicable.