What it is for
Proving that a prompt change did not break what already worked. Tuning a prompt is easy; noticing that the fix for today's case broke last week's is not — and with no test, the way you find out is the alert that stops making sense at three in the morning.
What a case is
A real input, the task it goes through, and what the answer must and must not say. Required is what has to show up; forbidden is what must not. Both are plain substrings — not a second model judging the first, which would be one more thing to trust.
A model grading another model fails in a way that is hard to see: it agrees with what sounds right. "The answer must contain the rejection code" is a question with one answer, and it costs nothing to check.
Running them
Run all sends every case through its task and shows the score. Each run spends real tokens, which is why nothing runs on its own: the moment to run them is right after changing a prompt, and the person who changed it is the one who knows.
A case with neither a required nor a forbidden phrase is not accepted: it would always pass, and a test that always passes is worse than no test.