Scientometrics Computer Science Applications
Evidence-constrained LLM reasoning framework: emerging technology evolution and rupture detection
Abstract
Emerging technologies often appear as weak, ambiguous signals in scientific literature, making them difficult to detect through conventional bibliometric indicators or expert screening alone. This study proposes an evidence-constrained Large Language Model (LLM) reasoning framework for tracing emerging technology evolution and detecting rupture-like changes in research activity. Using 351,167 arXiv computer science papers from 2020–2024 across ten subfields, we construct a rolling research ecosystem with 6-month windows and a 2-month step. Within each window, SPECTER2 embeddings and direct citation relations are combined to build a semantic–citation fused graph, and Leiden community detection is used to identify Research Activity Units (RAUs). Adjacent-window RAUs are connected through paper-level inheritance, enabling rupture signals to be quantified along four dimensions: scale, structure, impact, and novelty. External citation counts are incorporated into the impact analysis, and historical research backgrounds retrieved from OpenAlex and Crossref are used for novelty assessment. These lineage relations, rupture metrics, and semantic descriptions are then organized into structured evidence packs that constrain LLMs to generate standardized and auditable interpretations. Evaluation is conducted through full-sample comparison and expert-sample validation. Across 5,181 RAUs, evidence-constrained DeepSeek V3.2 and Kimi K2.6 are compared with a no-evidence DeepSeek baseline. In a 777-RAU expert evaluation, DeepSeek V3.2 achieves the highest agreement with expert judgments at 84.43%, followed by Kimi K2.6 at 70.46%, while the no-evidence baseline reaches only 32.05%. These results suggest that structured evidence constraints improve the consistency, interpretability, and verifiability of LLM-assisted scientometric analysis for emerging technology monitoring.