Distillation With a Reject Option

This evidence synthesis examines compression and abstention in one-step students used in safety-sensitive workflows. It argues that the appropriate object of evaluation is the trajectory from tokenization and draft generation through scoring, revision, compression, and release, not a model score in isolation. The review brings together the assigned studies with established work on uncertainty, robustness, provenance, and governance. Across these literatures, a common problem emerges: speed and fluency can conceal semantic loss, correlated self-evaluation errors, or domain-specific failure. The proposed framework separates evidence quality, model behavior, decision policy, and operational monitoring, then asks how each layer changes under distribution shift, adversarial pressure, or incomplete information. It recommends evaluation by slices and repeated trials, explicit reject and escalation policies, preservation of data and reasoning lineage, and prospective monitoring tied to defined actions. The result is a research agenda for systems that are efficient enough to use but also bounded enough to audit. No new experiment is claimed; the article develops a comparative conceptual model and identifies tests that would make future empirical claims more credible.

1. Zhang, Yin, et al. "SAINF: Intrinsic Self-Correction for Robust Machine Translation with Large Language Models." *Frontiers of Computer Science* (2026).
2. Tan, Wei, et al. "Evo-CuRL: Curriculum-Aware Reinforcement Learning over Code Lineage Graphs for Software Engineering Reasoning." *Proceedings of the 2026 International Conference on Multimedia Retrieval* (2026): 1327-1335.
3. Chen, Yiwei, et al. "One-Step Generative Distillation." *ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)* (2026).
4. Rei, Ricardo, et al. "COMET: A Neural Framework for MT Evaluation." *Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing*, 2020, pp. 2685-2702.
5. Wang, Xuezhi, et al. "Self-Consistency Improves Chain of Thought Reasoning in Language Models." *International Conference on Learning Representations*, 2023.
6. Ouyang, Long, et al. "Training Language Models to Follow Instructions with Human Feedback." *Advances in Neural Information Processing Systems*, vol. 35, 2022, pp. 27730-27744.
7. Rafailov, Rafael, et al. "Direct Preference Optimization: Your Language Model Is Secretly a Reward Model." *Advances in Neural Information Processing Systems*, vol. 36, 2023.
8. Sutton, Richard S., and Andrew G. Barto. *Reinforcement Learning: An Introduction*. 2nd ed., MIT Press, 2018.
9. Tishby, Naftali, Fernando C. Pereira, and William Bialek. "The Information Bottleneck Method." *Proceedings of the 37th Annual Allerton Conference on Communication, Control, and Computing*, 1999, pp. 368-377.
10. Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the Knowledge in a Neural Network." *NIPS Deep Learning and Representation Learning Workshop*, 2015.
11. Guo, Chuan, et al. "On Calibration of Modern Neural Networks." *Proceedings of the 34th International Conference on Machine Learning*, 2017, pp. 1321-1330.
12. Geifman, Yonatan, and Ran El-Yaniv. "Selective Classification for Deep Neural Networks." *Advances in Neural Information Processing Systems*, vol. 30, 2017.
13. Araci, Dogu. "FinBERT: Financial Sentiment Analysis with Pre-Trained Language Models." *arXiv preprint arXiv:1908.10063*, 2019.