Teacher Bias in Generative Distillation

This evidence synthesis examines bias inheritance in student models trained from powerful teachers. It argues that the appropriate object of evaluation is the trajectory from tokenization and draft generation through scoring, revision, compression, and release, not a model score in isolation. The review brings together the assigned studies with established work on uncertainty, robustness, provenance, and governance. Across these literatures, a common problem emerges: speed and fluency can conceal semantic loss, correlated self-evaluation errors, or domain-specific failure. The proposed framework separates evidence quality, model behavior, decision policy, and operational monitoring, then asks how each layer changes under distribution shift, adversarial pressure, or incomplete information. It recommends evaluation by slices and repeated trials, explicit reject and escalation policies, preservation of data and reasoning lineage, and prospective monitoring tied to defined actions. The result is a research agenda for systems that are efficient enough to use but also bounded enough to audit. No new experiment is claimed; the article develops a comparative conceptual model and identifies tests that would make future empirical claims more credible.

1. Zhang, Yin, et al. "SAINF: Intrinsic Self-Correction for Robust Machine Translation with Large Language Models." *Frontiers of Computer Science* (2026).
2. Li, Yuanhao, et al. "DRAFT-RL: Multi-Agent Chain-of-Draft Reasoning for Reinforcement Learning-Enhanced LLMs." *Proceedings of the AAAI Conference on Artificial Intelligence* 40.35 (2026): 29530-29537.
3. Chen, Yiwei, et al. "One-Step Generative Distillation." *ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)* (2026).
4. Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the Knowledge in a Neural Network." *NIPS Deep Learning and Representation Learning Workshop*, 2015.
5. Guo, Chuan, et al. "On Calibration of Modern Neural Networks." *Proceedings of the 34th International Conference on Machine Learning*, 2017, pp. 1321-1330.
6. Geifman, Yonatan, and Ran El-Yaniv. "Selective Classification for Deep Neural Networks." *Advances in Neural Information Processing Systems*, vol. 30, 2017.
7. Araci, Dogu. "FinBERT: Financial Sentiment Analysis with Pre-Trained Language Models." *arXiv preprint arXiv:1908.10063*, 2019.
8. Loughran, Tim, and Bill McDonald. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks." *Journal of Finance*, vol. 66, no. 1, 2011, pp. 35-65.
9. Gururangan, Suchin, et al. "Don't Stop Pretraining: Adapt Language Models to Domains and Tasks." *Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, 2020, pp. 8342-8360.
10. Vaswani, Ashish, et al. "Attention Is All You Need." *Advances in Neural Information Processing Systems*, vol. 30, 2017.
11. Sennrich, Rico, Barry Haddow, and Alexandra Birch. "Neural Machine Translation of Rare Words with Subword Units." *Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics*, 2016, pp. 1715-1725.
12. Papineni, Kishore, et al. "BLEU: A Method for Automatic Evaluation of Machine Translation." *Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics*, 2002, pp. 311-318.
13. Rei, Ricardo, et al. "COMET: A Neural Framework for MT Evaluation." *Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing*, 2020, pp. 2685-2702.