Pattern Analysis and Applications Computer Vision and Pattern Recognition

Hexa-p: physical plausibility-aware pre-training for pedestrian path prediction

Taketsugu, Hiromu, Nobuhara, Shohei, Ukita, Norimichi
Springer Nature · 2026-11-09 · DOI 10.1007/s10044-026-01804-x

Abstract

Pedestrian path prediction is a safety-critical component of autonomous vehicles and social robots that must navigate alongside humans in open urban environments. In such deployments, predictions that imply unnatural accelerations or impossible turns can directly trigger unsafe planning decisions, especially when only a few past frames of the pedestrian are observed. Because pedestrians are full-body humans whose paths are governed by articulated locomotion, we exploit two resources: 3D human body pose, which encodes locomotion-level constraints, and a humanoid physics simulator, which synthesizes virtually unlimited, physically executable pose-and-path data. We propose “Physical Plausibility-aware Pre-training for Pedestrian Path Prediction” (Hexa-P), which uses this synthesized data to jointly pre-train a path predictor and a plausibility estimator. Unlike prior work that uses a separately trained plausibility estimator, this joint pre-training improves both prediction accuracy and physical plausibility. While this simulation involves only a single agent, multiple agents mutually influence one another’s future paths in real-world scenes. We therefore introduce an adapter-based fine-tuning that preserves physical knowledge while adapting to scene-specific social path interactions. We also propose Target-relative Positional Encoding (TaPE) to bridge the gap between single-agent simulation and multi-agent path prediction. Experiments on JRDB, JTA, and the Urban dataset demonstrate accurate and plausible predictions of Hexa-P. Particularly in safety-critical short-observation scenarios, Hexa-P reduces prediction errors by up to 50% on the JRDB dataset compared with a strong pose-aware baseline.

Read at publisher →