llama.cpp cache makes host-held MoE experts faster to reuse
A proposed llama.cpp cache lets GPUs reuse selected Mixture-of-Experts (MoE) experts that remain in host memory, giving users with VRAM-limited cards a way to trade RAM for generation speed. In an AMD test, caching 67% of 23,981.25 MiB of host experts lifted warm token generation from 11.0 to 27.6 tokens per second, a 2.5x gain. The PR currently does not support multiple devices.