LoRA’s Low‑Rank Fine‑Tuning Crumbles on Multi‑Step Procedures
Researchers show that LoRA, the go‑to parameter‑efficient fine‑tuning method, consistently underperforms full fine‑tuning on procedural tasks like travel booking, Zoom support, and insurance claims. The weight updates needed to master these multi‑step workflows have effective ranks far above LoRA’s low‑rank limits, exposing a fundamental barrier for agentic LLM applications.