Databricks introduces prompt caching for open-source LLMs, boosting inference speed
Databricks now extends its built‑in prompt‑caching feature—previously limited to proprietary models—to open‑source LLMs on its platform. By reusing KV caches for repeated prompts, it cuts compute, latency, and cost for batch and real‑time inference, all without extra configuration.