Educational technology research and development Образование
Measuring cognitive presence in online discussions: automated detection and instructional insights from a MOOC context
Аннотация
This study develops and validates an interpretable transformer-based model for detecting cognitive presence (CP) in MOOC discussions to serve as a diagnostic foundation for instructional design and facilitation. Specifically, we built this model by fine-tuning the RoBERTa architecture with theory-derived structural cues from the Community of Inquiry framework to capture causal, synthesizing, and action-oriented language. Using data from six Social Work MOOCs (4,260 learners; 227,863 sentences), the model classified discourse into four CP phases and a fifth non-CP category. The cue-augmented RoBERTa achieved strong performance (macro F1 = 0.714; κ = 0.666), and SHAP analyses confirmed the model’s internal logic aligns with theory-consistent linguistic markers, ensuring the pedagogical transparency of the results. Learner-level hierarchical regressions revealed that while discussion quantity predicted behavioral persistence, Integration predicted motivation and satisfaction while Resolution predicted perceived knowledge transfer, functioning as diagnostic proxies that participation volume alone does not capture. These findings shift the role of CP analytics from descriptive observation to prescriptive action through a proposed Pedagogical Diagnostic & Engineering (PDE) framework. By mapping automated structural signals to specific instructional failure points, this framework offers concrete tactics for real-time scaffolding, such as Socratic prompts and practitioner spotlights, to move learners beyond exploratory monologues toward deeper knowledge construction and professional application.