Treat LLM judgments as features, not finished classifications
If you use an LLM to classify data, wrapping its verdict in a stock model can make the result easier to calibrate and tune. The post’s key move is to feed the LLM’s hard label into logistic regression, then estimate probabilities from labeled examples rather than trusting the model’s confidence. With a binary verdict, this reduces to empirical class proportions for examples receiving each label.