Knowledge and Information Systems Human-Computer Interaction

Pretrained language models with semantic embeddings for temporal knowledge graph completion

Lv, Shaoqing, Dong, Jungang, Wang, Xuanhong
Springer Nature · 2026-11-14 · DOI 10.1007/s10115-026-02932-w

Abstract

Temporal knowledge graph completion (TKGC) focuses on predicting missing facts in knowledge graphs whose entities and relations evolve over time, and plays an important role in applications such as intelligent question answering and event forecasting. Existing TKGC methods typically rely on graph structures and process temporal quadruples in isolation, which limits their ability to capture long-range temporal dependencies and rich semantic interactions across events. In this paper, we propose Temporal Semantic Language Model (TSemLM), a semantic-driven TKGC framework that avoids explicit graph construction and graph neural networks. TSemLM converts temporal quadruples into natural language texts and constructs time-ordered event sequences using a sampling strategy that jointly considers global frequency and temporal sensitivity, enabling effective modeling of contextual temporal semantics. A pretrained language model is then employed to extract context-aware semantic embeddings at the event level, which are used to predict missing entities or relations. By relying on semantic representations rather than structural encodings, TSemLM exhibits improved adaptability and generalization across temporal knowledge graphs. Extensive experiments on standard TKGC benchmarks demonstrate the effectiveness of the proposed approach. On the ICEWS14 and ICEWS18 datasets, TSemLM achieves state-of-the-art performance, raising the Mean Reciprocal Rank by 2.57 and 2.51 percentage points over the strongest baseline, corresponding to relative improvements of $$6.4\%$$ 6.4 % and $$9.1\%$$ 9.1 % , respectively. Additional experiments on ICEWS05-15 confirm that the gains are consistent across a broader temporal range and a substantially larger event scale, while component-wise ablations, prompt template robustness tests, and an efficiency analysis further characterize the behavior and cost of the proposed design.

Read at publisher →