Transformers can run a 180B MoE model from 32GB of RAM
A 180B-parameter mixture-of-experts model can now be pushed beyond a single GPU’s memory by streaming its experts directly from SSD, making local inference possible on much smaller machines. The approach keeps only the active experts in VRAM and moves the rest through RAM or disk; the post says token generation comes within 6% of Strata, while a related 32GB Mac setup reached 13.5 tokens per second with Qwen3.8-Flash-Next.