Hybrid LLMs Beat Transformers on Meaningful Tokens, Lose on Exact Repeats
AllenAI’s Olmo Hybrid outperforms its pure‑transformer sibling on tokens that carry semantic weight—nouns, verbs, adjectives, and coreferential pronouns—while the transformer wins on verbatim copy‑tasks. The gap stems from the hybrid’s recurrent layers, which excel at sequential reasoning but sacrifice exact recall. This shows hybrid architectures trade memory for deeper understanding.