Argumentation Лингвистика и язык
The Implicit Audience Problem in Computational Argument Quality Assessment: A Pragmatic Critique and the ASAQ Framework
Аннотация
This paper identifies a structural tension in computational argument quality assessment. Although the influential framework of Wachsmuth et al. (2017a, b) explicitly invokes a target audience in its dimensional definitions, its annotation protocols leave that audience operationally unspecified. A recent observation by Ivanova and Gubelmann (2025) reveals high within-annotator agreement between logical and dialectical dimensions alongside only moderate inter-annotator agreement. We argue that this pattern is best explained by the implicit audience hypothesis: when no audience is specified, human annotators unconsciously project one, producing internal coherence within each annotator while generating divergence across annotators. Drawing on four pragmatic traditions—Perelman and Olbrechts-Tyteca’s New Rhetoric, Eco’s Model Reader, Clark’s common ground, and Sperber and Wilson’s relevance theory—we test the hypothesis through qualitative pragmatic analysis of eighty argument pairs from the IBM-Rank-30k corpus. The analysis shows that the human labels are consistently interpretable as quality relative to a projected audience: the same annotator’s judgements are best made sense of by attributing different audiences across topics and even within a single topic, and a recurring evaluative profile that can be read as broadly liberal appears across many pairs. We propose ASAQ (Audience-Sensitive Argument Quality), a reconstruction of the Wachsmuth framework grounded in three principles: explicit audience specification, multiple-audience modelling, and pragmatic calibration of the three dimensions. We outline concrete computational implementation pathways through audience-conditioned prompting and fine-tuning of large language models (LLMs), and discuss implications for LLM-as-judge methodology and future corpus design.