Transformers brings smaller llama.cpp models to Apple Silicon Macs
Transformers can now load GGUF, llama.cpp’s local-model format, so Apple Silicon Mac users can run quantized checkpoints without switching inference stacks. GGUF bundles weights and metadata in one file, while quantization trades precision for memory: the page’s Qwen3.5-4B example drops from 8.42 GB in BF16 to 2.74 GB in Q4_K_M. Support currently targets Qwen3.5 and the latest Transformers code.