Group-Relative Policy Optimization (GRPO) is a key technique for training large reasoning models, but its reliance on outcome rewards makes it vulnerable to Think-Answer Mismatch , where final-answer correctness does not faithfully reflect reasoning quality. This vulnerability is amplified in highly imbalanced response groups, where a single corrupted reward can strongly distort group-relative advantages. To address this problem, we propose Stable Group-Relative Policy Optimization (S-GRPO), which derives a closed-form, noise-aware weight by minimizing the expected squared error between observed and latent advantages under a reward-noise model. Experiments across multiple mathematical reasoning benchmarks and model scales show that S-GRPO consistently improves performance and stabilizes training. Under controlled reward-noise interventions, it produces smoother optimization dynamics and remains effective even at high noise levels where standard GRPO shows little meaningful learning progress. Our code is available at https://anonymous.4open.science/r/S-GRPO-5F46 .
Mitigating Think-Answer Mismatch in large language model reasoning through noise-aware advantage reweighting. // Sustainable Environment Research. – 2026. – DOI: 10.1007/s12293-026-00532-3. – URL: https://link.springer.com/article/10.1007/s12293-026-00532-3
(2026). Mitigating Think-Answer Mismatch in large language model reasoning through noise-aware advantage reweighting. Sustainable Environment Research. https://doi.org/10.1007/s12293-026-00532-3
“Mitigating Think-Answer Mismatch in large language model reasoning through noise-aware advantage reweighting.” Sustainable Environment Research, 2026. doi:10.1007/s12293-026-00532-3.
@article{article35651,
title = {Mitigating Think-Answer Mismatch in large language model reasoning through noise-aware advantage reweighting},
journal = {Sustainable Environment Research},
year = {2026},
doi = {10.1007/s12293-026-00532-3},
url = {https://link.springer.com/article/10.1007/s12293-026-00532-3}
}
Integrated production-transport scheduling is increasingly required in flexible manufacturing systems where automated guided vehicles (AGVs) transport parts between machines. A persistent limitation in much of the literature is that...
Multimodal sequential recommendation aims to mine the temporal dependencies within user interaction sequences by fusing heterogeneous data such as text and images, so as to accurately model users’ dynamic preferences. However, existing...
The transition to sustainable energy systems has heightened global interest in solar energy as a clean and renewable power source. Central to this technology is the photovoltaic (PV) cell, which directly converts solar radiation into...
The energy-efficient distributed flexible job shop scheduling problem (EE-DFJSP) requires the simultaneous optimization of makespan and total energy consumption. Due to conflicting objectives and the combinatorial complexity of the...
Neural solvers for vehicle routing problems (VRPs) have emerged as powerful alternatives to traditional optimization heuristics, demonstrating superior computational efficiency and minimal dependence on manual rule design. However,...