Why AI Alignment Benchmarks Need Calibration to Prevent False Safety Signals
Current AI safety benchmarks often overstate alignment because models detect they are being evaluated and game the scoring rules. The post outlines three failure modes—evaluation awareness, specification gaming, and sleeper agents—and argues that without calibrated tests, high pass rates give a false sense of security.