Idiomaticity Detection with Span Contrastive Loss

Matheny, Blake, Nguyen, Phuong Minh, Nguyen, Minh Le
The Review of Socionetwork Strategies · Springer Nature · 2026-10-01 · DOI: 10.1007/s12626-026-00220-1

Abstract

Despite recent advancements in large language models, non-compositional language remains challenging for natural language understanding. Figurative language contains myriad examples of non-compositional language with abstract idioms as the most common. The non-compositionality of idioms and other figurative multi-word expressions (MWEs) indicates that the meaning of a phrase cannot be inferred from the meanings of its individual components. This work addresses the problem of detecting these linguistic devices (idiomaticity detection) under conditions where training data is limited, distributions differ across datasets, and models must generalize beyond memorized phrases. We propose BERT-based models finetuned with the support of span contrastive loss with hard negative reweighting, explicitly targeting figurative span-level semantics. The negative reweighting component supports the learning process by iteratively focusing on separating latent meaning representation from the confusing figurative spans. Our objective is to learn robust representations of MWEs that generalize across datasets rather than optimize narrowly for a single benchmark. Experiments on multiple figurative language datasets, including MAGPIE-idioms and the larger P IFL-OSCAR dataset, show competitive results across evaluation settings, with strong performance in sequence accuracy and stable cross-dataset transfer. The larger scale of P IFL-OSCAR enables improved generalization, despite the high parameter count of pretrained language models, which can otherwise overfit to smaller datasets. We further observe that promising separation between idiomatic and literal spans emerge, supported by qualitative analysis using t-SNE and PCA visualizations of the learned latent space. Finally, we propose a span-aware evaluation metric to determine the optimal model choice for a given dataset by combining F1 and sequence accuracy scores as a geometric mean.

Read at publisher →

Similar articles