Journal of New Approaches in Educational Research Education
Development and validation of an adaptive learning readiness assessment framework for Moodle courses
Abstract
Abstract Purpose While Moodle is widely adopted in higher education, institutions struggle to leverage its features for adaptive learning. This study develops and validates the Adaptive Learning Readiness Assessment Framework (ALRAF), a course-level diagnostic instrument for evaluating a Moodle course’s structural capability to support adaptive learning experiences. Design We employ a quantitative cross-sectional design coupled with a novel Multi-LLM Synthetic Expert Consensus (MLSEC) protocol for content-validity evidence. ALRAF was developed through literature synthesis grounded in the ICAP framework (Chi and Wylie, Chi, Educational Psychologist 49 , 2014) and validated by a 40-panelist synthetic expert panel constructed across eight large-language-model providers and five stratified expert personas using two pre-registered Delphi rounds with falsifiable decision rules ( $$I$$ -CVI $$\geq$$ 0.78, $$V$$ -Aiken $$\geq$$ 0.70, modified $$\kappa^{*} \geq 0.74$$ ). The validated framework was applied to a Moodle 3.8.2 dataset of 985 courses delivered at Kryvyi Rih State Pedagogical University (Ukraine) across 2020–2022. Findings The synthetic panel converged on six dimensions: Content Variety, Interaction Diversity, Assessment Flexibility, Learning Path Personalization, Feedback Mechanisms, and the panel-proposed AI & Data-Driven Adaptivity Integration (ADAI). The six-dimensional $$\textrm{ALRS}_{6}$$ correlated significantly – but negatively – with the proportion of high grades ( $$A+B$$ ): Pearson $$r=-0.22$$ , and positively with low grades $$(E+F): r=+0.22$$ (both $$p < .001$$ ). This inverse relationship runs counter to what would be expected if ALRAF directly indexed pedagogical quality, and reframes ALRAF as a measure of structural readiness rather than learning effectiveness; we interpret the sign in Sect. 6 in terms of course-difficulty and compensatory-engineering effects. Faculty differences were significant (ANOVA $$F(8, 976)=10.26$$ , $$p < .001$$ , $$\eta^{2}=0.078$$ , $$\omega^{2}=0.070$$ ). A multiple-regression model controlling for educational level, form of education, and faculty achieved adjusted $$R^{2}=0.18$$ ( $$F(19, 965)=12.10$$ , $$p < .001$$ ). The framework reveals strong implementation of content variety but near-zero readiness in learning-path personalization and AI integration – itself a notable institutional finding. Contribution Methodologically, the study introduces MLSEC as a transparent AI-augmented approach to rubric content validation, with synthetic-panel limitations explicitly disclosed. Substantively, ALRAF provides a replicable structural-readiness index whose correlations with student outcomes are non-trivial in direction and magnitude; it helps institutions identify capability gaps (especially around AI integration) without claiming to forecast student success.