New LLM Architectures Slash Long-Context Costs with KV Sharing and Compressed Attention
Sebastian Raschka surveys the latest open‑weight LLM releases – Gemma 4, DeepSeek V4, ZAYA, Laguna XS – highlighting architecture tricks like KV‑sharing, multi‑head compression and compressed attention that cut long‑context costs. These design tweaks shrink KV caches, lower memory traffic, and make token‑heavy reasoning and agent workflows more efficient.