A Fairness Audit Finds CLIP Nearly Blind to Museum-Wide Gender Effects
A controlled audit of 1,500 Metropolitan Museum objects found no statistically significant gender effect in OpenAI CLIP or OpenCLIP scores for “masterpiece,” “quality,” and “influence.” But the result is less reassuring than it sounds: the models explained under 2% of score variance, suggesting these broad prompts are too blunt to detect localized bias. Archival metadata and the 41.2% of unattributed works remain major confounders.