Memory Recurrent Units give Transformers linear‑time memory without attention
The new paper introduces Memory Recurrent Units (MRUs), a family of RNNs that blend persistent memory from nonlinear dynamics with the parallel‑scan efficiency of state‑space models. By leveraging multistability, the bistable MRU runs in linear time while retaining long‑range information, offering a viable, low‑cost alternative to attention‑heavy Transformers.