RL Fine-tuning Preserves LLM Internal Circuits Better Than Supervised Methods
The paper shows RL fine‑tuning retains prior capabilities by preserving internal computational circuits, unlike supervised fine‑tuning which adapts faster but disrupts circuits and causes forgetting. Using a metric called differential circuit vulnerability on Qwen2.5‑3B‑Instruct for scientific QA, they demonstrate RL’s mechanistic advantage.