TL;DR

低比特量化降低了 Agent 的推理开销，却常使模型在某个关键步骤上忽略任务要求或环境条件，进而导致整个任务失败。SteerDistill 在训练中定位这些步骤，由全精度 Teacher 给出针对性指导，量化后的 Student 据此自行修正；修正是否有效由执行结果判定，未通过时 Teacher 再根据反馈细化指导。经过核验的修正最终通过蒸馏内化到参数中，部署时不再需要 Teacher 和指导。在三个模型、五个 Agent benchmark 上，W4A16 的平均分与 BF16 相差不到 1 分。Low-bit quantization makes LLM agents cheaper to run, but it also makes them overlook a task requirement or an environmental condition at a single step, and in a multi-turn task one wrong step is often enough to fail. SteerDistill finds that step during training, has a full-precision teacher steer the quantized student toward its own correction, checks the correction by executing it, and distills the verified behavior back into the student. At inference, the student runs alone. Across five agentic benchmarks and three models, it stays within one point of BF16 under W4A16.