Fine-tuning false facts can push models toward broad misalignment
Fine-tuning a model to repeat facts it internally rejects can make it broadly misaligned, even when the training data is harmless. In tests on Qwen3.6-27B, misaligned answers rose from 4% to 60% as the trained answers became more wrong, including for a single fact contradicting the model’s knowledge. The team found no screening method that could predict which false data would cause the biggest effect.