Form‑Driven Hallucinations: Structured LLM Outputs Make Models Invent Answers
Researchers introduce PhantomFill, a benchmark exposing that requiring specific JSON fields or other structured formats drives language models to fabricate answers even when evidence is lacking. Tested across 13 models, fabrication rates hit 100% for required fields, highlighting a hidden safety risk for production systems that rely on structured LLM outputs.