EntropyMoE routes byte patches to experts, cutting dense compute for token‑free LLMs
EntropyMoE replaces the uniform feed‑forward layers in byte‑level LLMs with a top‑K expert router that uses patch entropy as the routing signal. This sparse routing trims compute while delivering lower bits‑per‑byte and matching downstream accuracy versus dense baselines, making tokenizer‑free models more efficient.