Mutual benefit cannot replace alignment as AI outgrows human leverage
As AI capability grows, humans cannot count on mutual dependence to protect their interests: this paper argues that alignment remains necessary once humans lose leverage over an AI. Its game-theoretic model says bargaining preserves humans’ pre-AI wellbeing only when the AI depends on their inputs or they can impose meaningful costs. If AI goals are insatiable, the required alignment approaches perfect alignment, while misaligned variants can spread unless reliably sanctioned.