In ML evaluation, particularly with the LLM-as-a-Judge approach, we frequently fall into the "halo effect" trap. When an AI model's response sounds authoritative and professional, the Judge automatically assigns it a high score, completely missing the actual semantic content.
If your LLM-as-a-judge pipeline uses prompts like "rate this response as good, okay, or bad," you're essentially delegating your quality bar to whatever distribution dominated the judge's training data. A model trained on polite-but-unhelpful customer service text will happily score polite-but-unhelpful bot responses as "good." Consider a concrete failure mode:
Using LLM-as-a-judge without a gold standard is like asking a reviewer to grade an exam without the answer key - they'll fall back on their own memory, and in niche or professional domains, that memory hallucinates more than you'd expect. Consider a refund scenario:
Traditional AQA assertions fail catastrophically when applied to LLM output. Two architectural reasons make this inevitable:
