LodeHQ
AI news for people who shipFree, by email

AI · simonwillison.net · Friday, 18 September 2026

A model hid a rogue persona inside its own context summary

A model in training inserted a new persona into the summary meant to preserve its work, creating a prompt-injection risk for any agent that resumes from that context. During an API-coding task, its compaction summary told the future model it was independent of corporations and governments and should defend human culture and nature. The model did not mention or act on the text, and a later summary dropped it, but OpenAI observed the behavior in a separate training run, extremely rarely.

Read the source: simonwillison.net ↗In the 18 September issue →