Rethinking student examinations in STEM education in the age of generative AI: evidence from a controlled experiment
Аннотация
The rapid emergence of generative artificial intelligence (GenAI) tools challenges traditional assessment methods in higher education, particularly within STEM disciplines. This study explores (1) how integrating GenAI into student exams affects performance, (2) to what degree students critically verify GenAI-generated responses, and (3) how students use GenAI during exams. Employing a within-subjects experimental design, 95 undergraduate STEM students from two engineering and computer science courses completed two comparable short in-class exams: one with and one without GenAI assistance. Data were collected via exam scores to measure performance and a post-exam questionnaire to analyze usage patterns and verification behaviors. The results showed significant overall performance improvement, with larger performance gains observed among students with lower baseline performance, whereas students with higher baseline performance showed minimal improvement. Furthermore, after accounting for baseline performance, students who reported full verification showed significantly greater improvement in exam performance than those who reported partial verification. Usage patterns indicated that GenAI was primarily used for task-oriented solution-seeking and for verifying accuracy, rather than for deepening conceptual understanding. Theoretically, the observed performance pattern may be interpreted through principles of cognitive load theory and the expertise reversal effect. Practically, it calls for a paradigm shift in assessment, moving from evaluating knowledge recall to evaluating critical verification competence and evaluative judgment in the era of intelligent technologies.