Journal on Multimodal User Interfaces Обработка сигналов

Design and implementation of gesture-based human computer interaction for visually impaired people: a cognitive load perspective

Lohar, Durgesh, Sen, Bibhash, Basu, Anupam
Springer Nature · 2026-08-01 · DOI 10.1007/s12193-026-00483-6

Аннотация

Human-computer interaction exploits various input methods to improve user experience, but those who don’t have visual access suffer more than the mainstream. Hand gestures are becoming increasingly popular in the design of assistive technologies, representing a natural means of communication. In this context, this paper proposes a novel Gesture-based Computer Interaction (GCI) system to facilitate computer interaction for visually impaired people. However, a large set of gestures for a simple application introduces complexity, which poses challenges for the visually impaired to interact with computers. Therefore, an accessible assistive application (keyboard) with a minimal set of gestures is designed here. The performance of the proposed GCI is evaluated from a cognitive load perspective. Nineteen (19) participants engaged with the proposed GCI in various cognitive scenarios with the computer. Several dimensions are evaluated, including physiological measures (skin conductance), NASA TLX, and performance indicators (response time, gesture response time, false responses, forgot word counts). Among all metrics, the gesture response time revealed that the proposed GCI technique had a 39% faster gesture response time than the existing technique. In addition, the galvanic skin response revealed a marginal reduction in skin conductance (low cognitive load), which means that the proposed GCI technique caused a more relaxed reaction than the existing technique. The proposed GCI technique significantly outperformed the existing technique in almost all key areas (performance, physiological, and NASA-TLX), which opens an emerging multimodal interface in the design and development of assistive technologies for visually impaired people.

Читать у издателя →