Integer-only inference keeps the same LLM run identical everywhere
DetLLM replaces floating-point operations with integer arithmetic so the same prompt can produce bit-identical output across CPUs, GPUs, vendors, and batch sizes. Floating-point sums change when hardware kernels add values in a different order, which can alter the winning next token: one cited test got 80 answers from 1,000 temperature-zero runs. In the author’s demo, fp16 runs diverge by token 457 across nine configurations, while the integer runs stay aligned.