LLMs Give “Likely” a Different Meaning Than Humans Do
LLMs’ verbal confidence markers do not map to uncertainty the way human phrases such as “possible” and “likely” do, making natural-language confidence harder to interpret safely. The paper benchmarks models against a corpus of human uncertainty markers, then fits a marker-to-probability mapping directly to model outputs rather than relying on repeated sampling. That exposes systematic gaps between model confidence semantics and human expectations.