Attention manifolds let queries steer what models extract
A new attention mechanism lets a query change not only how strongly a language model reads a token, but which parts of that token’s value it extracts, giving researchers a small, editable control surface for steering outputs. The learned 2D B-spline surfaces add 0.3% parameter overhead and cut WikiText-2 perplexity by 2 to 2.5 points on LLaMA 3.2 models. Editing one layer changed greedy output on 9 of 10 prompts, while setting coefficients to -1 created “attention walls” that redirected an explosive-device prompt toward general education.