Bootstrapped On-Policy Distillation (BOPD)
Under review at ICLR 2027.
Overview
A two-stage framework bootstraps a task-adapted teacher through reference-conditioned self-distillation, then trains an independent student via on-policy distillation. On Qwen3-4B, mean mathematical reasoning accuracy improves from 18.06% to 23.98%, surpassing the constructed teacher.