TRSP Architecture Stops Representation Collapse, Boosts LLM Long-Context Accuracy
The paper pinpoints representation collapse as the core obstacle for long-context language models, showing that existing fixes swing between homogenization and isolation extremes. Their Topologically Regularized Side-Path (TRSP) injects a parameter-free triangular box and length-aware gate to balance spectral mixing and rank, delivering up to 50-point gains on extended benchmarks.