An interpretable modality-attention transformer architecture for multispectral feature fusion in automated wood species identification

Wang, Jinsong, Wang, Yuan, Avramidis, Stavros
Wood Science and Technology · Springer Nature · 2026-11-12 · DOI: 10.1007/s00226-026-01826-y

Abstract

This paper proposes a multispectral feature fusion framework for multispectral wood identification using spectral data acquired by near-infrared (NIR) spectroscopy, hyperspectral imaging (HSI), and terahertz (THz) spectroscopy. For five wood species, the framework combines spectral preprocessing with a three-stage filter–wrapper–embedded feature selection strategy, retaining 13 HSI, 9 NIR, and 22 THz features. To exploit complementary spectral information, this paper develops weighted feature fusion and a modality-attention Transformer fusion (MATF) model incorporating modality-level attention for adaptive cross-modal interaction. Weighted fusion achieved a mean classification accuracy (ACC) of 92.6% ± 4.6% and a mean macro-averaged F1 (Macro-F1) of 92.5% ± 4.7%. Under stratified five-fold cross-validation repeated three times, MATF achieved a mean ACC of 94.46% ± 4.98% and a mean Macro-F1 of 94.43% ± 5.19%. An ablation study showed that removing the Transformer reduced mean ACC and Macro-F1 to 92.13% and 91.59%, respectively, supporting its contribution to classification performance. The framework further incorporates modality-level Shapley additive explanations (SHAP) to interpret the attribution of each spectral source to model predictions. These results support the effectiveness of combining complementary spectral information with adaptive cross-modal interaction for wood species identification​.

Read at publisher →

Similar articles