Cognitive Research: Principles and Implications Когнитивная нейронаука

Improving human performance in deepfake detection: limits of aggregation and human–AI collaboration

Kramer, Robin S. S., Lister, Victoria P. M.
Springer Nature · 2026-10-05 · DOI 10.1186/s41235-026-00758-2

Аннотация

Given that deepfake videos are now easy to generate and increasingly high in quality, the present research investigated human detection performance and several routes to improvement. Across four experiments, participants demonstrated above-chance detection accuracy when viewing YouTube clips and their corresponding deepfakes, but performance remained modest and error-prone. In Experiment 1, aggregating responses across the full sample produced reliable improvements in accuracy, consistent with a ‘wisdom of the crowd’ effect. However, in Experiment 2, participants showed no evidence of metacognitive sensitivity. Although confidence was modestly higher for correct than incorrect decisions, it did not reliably discriminate between them at the trial level. Consistent with this finding, confidence-weighted fusion of participant pairs reduced rather than improved accuracy. In Experiments 3 and 4, we examined human–AI collaboration, with participants making decisions before and after receiving simulated algorithmic guidance. Performance improved following guidance when it was highly accurate, but this benefit was offset by a tendency to follow incorrect advice when it occurred. Notably, this reliance on simulated algorithmic guidance was unaffected by the prevalence of deepfakes within the stimulus set. Taken together, these findings suggest that, while both aggregation and human–AI collaboration can enhance performance under certain conditions, their effectiveness is constrained by poor metacognitive insight and a general tendency to align decisions with external advice. Improving deepfake detection may depend not only on advances in algorithmic performance, but also on a better understanding of how humans interpret and use both perceptual and algorithmic information. As deepfake videos become easier to create and more difficult to distinguish from genuine footage, improving human detection performance is an increasingly important societal challenge. Across four experiments, we evaluated several potential solutions and found that combining decisions across large groups can increase accuracy, whereas confidence-based approaches are ineffective because people have little insight into when their judgements are correct. We also found that simulated algorithmic guidance can improve performance when highly reliable but people readily follow incorrect advice when it is provided. These findings highlight both the promise and the limitations of human- and AI-based approaches to deepfake detection and suggest that effective decision-support systems must account for the ways in which people interpret and rely on both their own judgements and external guidance.

Читать у издателя →