publications

2026

1. 

   When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning

   Xiaogeng Liu, Xinyan Wang, Yingzi Ma, Yechao Zhang, and Chaowei Xiao

   In Advances in Neural Information Processing Systems (NeurIPS), 2026

   arXiv Code PDF Bib Abs

   On-policy self-distillation (OPSD) trains a student on its own rollouts using a privileged teacher, but its standard objective weights all generated tokens equally, implicitly treating the privileged teacher target as equally reliable at every student-visited prefix. Existing entropy-based OPD methods relax this uniformity by modulating token-level supervision with teacher entropy, but high teacher entropy in reasoning has an ambiguous reliability meaning: it can reflect either non-viable uncertainty or benign solution diversity. To identify this phenomenon, we introduce a branch-viability diagnostic. Specifically, we record next-token alternatives from the privileged-answer teacher prompt, force each alternative after the student prompt plus its on-policy spine prefix, and test whether the resulting student-template continuation recovers the correct answer. On Qwen3-4B, we find that an oriented within-sequence position score is the strongest tested predictor of teacher-token reliability, reaching an area-under-ROC-curve (AUROC) of 0.83 with a 95% cluster-bootstrap interval of [0.66, 0.95]; local uncertainty scores are at most 0.57. Motivated by this trajectory-level structure, we propose Position-Weighted On-Policy Self-Distillation (PW-OPSD), which applies an increasing position weight while keeping the same student rollout, privileged teacher pass, and clipped forward-KL target as OPSD. In our comprehensive evaluations with different random seeds, the diagnostic-derived PW-OPSD improves AIME 2024 and AIME 2025 Avg@12 by +1.0 and +1.1 points, and a generalization evaluation on two larger-scale models from different families, DeepSeek-R1-Distill-Llama-8B and Olmo-3-7B-Think, also demonstrates consistent aggregate Avg@12 improvements. These results show that teacher-token reliability in reasoning distillation is trajectory-structured and can be utilized without additional teacher computation. The code is available at https://github.com/SaFo-Lab/PW-OPSD.

   @inproceedings{liu2026pwopsd,
     title = {When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning},
     author = {Liu, Xiaogeng and Wang, Xinyan and Ma, Yingzi and Zhang, Yechao and Xiao, Chaowei},
     booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
     year = {2026},
     eprint = {2605.21606},
     archiveprefix = {arXiv},
     primaryclass = {cs.LG},
   }

2. 

   ReasoningBomb: A Stealthy Denial-of-Service Attack by Inducing Pathologically Long Reasoning in Large Reasoning Models

   Xiaogeng Liu, Xinyan Wang, Yechao Zhang, Sanjay Kariyappa, Chong Xiang, Muhao Chen, G. Edward Suh, and Chaowei Xiao

   In Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2026

   Website arXiv Code PDF Bib Abs

   Large reasoning models (LRMs) extend large language models with explicit multi-step reasoning traces, but this capability introduces a new class of prompt-induced inference-time denial-of-service (PI-DoS) attacks that exploit the high computational cost of reasoning. We first formalize inference cost for LRMs and define PI-DoS, then prove that any practical PI-DoS attack should satisfy three properties: (1) a high amplification ratio, where each query induces a disproportionately long reasoning trace relative to its own length; (ii) stealthiness, in which prompts and responses remain on the natural language manifold and evade distribution shift detectors; and (iii) optimizability, in which the attack supports efficient optimization without being slowed by its own success. Under this framework, we present ReasoningBomb, a reinforcement-learning-based PI-DoS framework that is guided by a constant-time surrogate reward and trains a large reasoning-model attacker to generate short natural prompts that drive victim LRMs into pathologically long and often effectively non-terminating reasoning. Across seven open-source models (including LLMs and LRMs) and three commercial LRMs, ReasoningBomb induces 18,759 completion tokens on average and 19,263 reasoning tokens on average across reasoning models. It outperforms the runner-up baseline by 35% in completion tokens and 38% in reasoning tokens, while inducing 6-7x more tokens than benign queries and achieving 286.7x input-to-output amplification ratio averaged across all samples. Additionally, our method achieves 99.8% bypass rate on input-based detection, 98.7% on output-based detection, and 98.4% against strict dual-stage joint detection.

   @inproceedings{liu2026reasoningbomb,
     title = {ReasoningBomb: A Stealthy Denial-of-Service Attack by Inducing Pathologically Long Reasoning in Large Reasoning Models},
     author = {Liu, Xiaogeng and Wang, Xinyan and Zhang, Yechao and Kariyappa, Sanjay and Xiang, Chong and Chen, Muhao and Suh, G. Edward and Xiao, Chaowei},
     booktitle = {Proceedings of the ACM Conference on Computer and Communications Security (CCS)},
     year = {2026},
     eprint = {2602.00154},
     archiveprefix = {arXiv},
     primaryclass = {cs.CR},
   }

3. 

   ROM: Real-time Overthinking Mitigation via Streaming Detection and Intervention

   Xinyan Wang, Xiaogeng Liu, Ming Pei, and Chaowei Xiao

   Submitted to ACL Rolling Review (ARR), Aug 2026

   Website arXiv Code PDF Bib Abs

   Large Reasoning Models (LRMs) often reach a correct solution before their long Chain-of-Thought trace ends, yet continue with redundant verification, repeated attempts, or unnecessary exploration that wastes computation and can even overturn the correct answer. We frame this behavior as a latent productive-to-redundant transition and show it is directly reflected in hidden states: around first-correct-solution (FCS) boundaries, late-layer representations separate efficient from overthinking tokens, while boundary-permutation and position controls collapse. We propose ROM, a streaming intervention framework that monitors a frozen LRM with a lightweight hidden-state detector (roughly 0.1% of backbone parameters) and intervenes at well-formed reasoning boundaries; Counterfactual Self-Correction (CSC) balances supervision with wrong-to-correct trajectories, preserving useful pre-FCS self-correction. Unlike prior adaptive early-exit methods, ROM extracts no intermediate answers, launches no probe decoding, and updates no backbone weights. Across five backbones from three model families and five reasoning benchmarks, against ten recent baselines under a shared protocol, ROM with CSC attains the highest accuracy in 19 of 25 model–benchmark settings, cuts response length by 28–77% (mean 45%) versus vanilla decoding, and is the only method on the accuracy–length Pareto front in every setting. The same MATH500-trained supervision transfers zero-shot across scales, families, and task domains, and end-to-end wall-clock latency drops by 46.5% with roughly 5% per-token overhead. The code is available at https://github.com/SaFo-Lab/ROM.

   @article{wang2026rom,
     title = {ROM: Real-time Overthinking Mitigation via Streaming Detection and Intervention},
     author = {Wang, Xinyan and Liu, Xiaogeng and Pei, Ming and Xiao, Chaowei},
     journal = {Submitted to ACL Rolling Review (ARR)},
     month = aug,
     year = {2026},
     eprint = {2603.22016},
     archiveprefix = {arXiv},
     primaryclass = {cs.AI},
   }

2023

1. 

   MLE with datasets from populations having shared parameters

   Jun Shao, and Xinyan Wang

   Statistical Theory and Related Fields, 2023

   HTML PDF Bib Abs

   We consider maximum likelihood estimation with two or more datasets sampled from different populations with shared parameters. Although more datasets with shared parameters can increase statistical accuracy, this paper shows how to handle heterogeneity among different populations for correctness of estimation and inference. Asymptotic distributions of maximum likelihood estimators are derived under either regular cases where regularity conditions are satisfied or some non-regular situations. A bootstrap variance estimator for assessing performance of estimators and/or making large sample inference is also introduced and evaluated in a simulation study.

   @article{shao2023mle,
     title = {MLE with datasets from populations having shared parameters},
     author = {Shao, Jun and Wang, Xinyan},
     journal = {Statistical Theory and Related Fields},
     volume = {7},
     number = {3},
     pages = {213--222},
     year = {2023},
     doi = {10.1080/24754269.2023.2180185},
   }