LodeHQ
AI news for people who shipFree, by email

AI · lesswrong.com · Monday, 16 November 2026

A TASTE shortcut makes stronger models look worse than chance

A simple term-counting rule reaches 63.0% on Anthropic’s TASTE benchmark, while stronger models fall below chance once that shortcut is removed, raising questions about what the test measures. The rule picks the proposal with fewer mechanistic-interpretability terms, a pattern that separates 40 of the 92 proposal pairs. On the remaining 52 pairs, OpenAI models average 29.6%, with 8 of 9 significantly below chance.

Read the source: lesswrong.com ↗In the 16 November issue →