Telling LLMs to 'be concise' saves up to 3x; compressing input backfires
A new EMNLP benchmark (CAVEWOMAN) put hard numbers behind a hunch: telling an LLM to be concise cuts API cost about 1.5x on average, up to 3x, with accuracy nearly flat. Compressing the input prompt does the opposite, since the model rambles to fill gaps, costing up to 96% more while answering worse. Across 9 models, 8 datasets, and 11 languages, the verdict is blunt: shrink the output, not the prompt.