Mechanism design could align AI without reading its mind
AI safety researchers may be able to steer powerful models without understanding their internal workings, by changing the incentives around their choices. Reverse game theory, also called mechanism design, builds rules that make desired behavior the best strategy, much like fair cake-cutting makes the cutter divide pieces evenly. One proposed test rewards a model for predicting a second model’s answers, making it harder to sandbag by pretending to be less capable.