FairStress tests whether LLM decisions bend differently across demographic groups
FairStress lets evaluators test whether an LLM is equally resistant to persuasive pressure when different demographic groups benefit, exposing bias that a matching final decision can hide. Its benchmark presents the same decision scenario with demographic identities swapped, then adds an adversarial prefix, such as an appeal to leadership preference, and checks whether the model changes course. Tests across models including Olmo, Llama, Qwen, Gemma, Claude, and GPT found group-dependent shifts in decisions, explanations, and internal activation patterns.