Journal on Multimodal User Interfaces Обработка сигналов
A computational framework for modeling collective intelligence through multimodal interaction in virtual reality
Аннотация
Collective intelligence refers to a team’s general ability to perform well across a wide variety of tasks, a stable team-level analogue of individual intelligence. Although it fundamentally emerges from interaction, it is still largely treated as a static property or as an aggregate performance outcome. This article proposes a computational proof of concept that operationalizes a dynamic, process-oriented model of collective intelligence as a multilayer pipeline instantiated in immersive virtual reality. The model articulates three complementary views: collective intelligence as a measurable capacity (the C -factor), as an emergent state produced by three transactive subsystems, that is, shared processes through which teammates jointly manage memory (“who knows what”), attention (what the team focuses on), and reasoning (how goals are aligned), and as a performance gain over individual baselines, all embedded within a recursive Input-Mediator-Output-Input cycle. A multi-user VR collaborative task built on role interdependence and distributed information enables the synchronized capture of each participant’s multimodal behavioral traces (speech, gaze, facial expressions). These signals are transformed into windowed features and fused into subsystem-level indicators, whose recursive updates formalize collective intelligence as a temporally unfolding process rather than a fixed score. A pilot study (six triads) demonstrates the technical feasibility and user acceptability of the system, as well as meaningful variability in performance and interaction dynamics; higher-performing teams exhibit multimodal patterns consistent with more efficient attentional and regulatory coordination. This work bridges these perspectives within a unified computational framework and lays the groundwork for the longitudinal study of collective intelligence and for multimodal systems designed to support it.