Suppose an AI agent reviews a loan application and recommends declining it.
You start by checking the ground truth. The applicant should, in fact, have been declined, so the agent gets a green checkmark.
Pass.
Then, you look one level deeper. It turns out the agent based its recommendation on the applicant’s ZIP code instead of the relevant credit information.
Still a pass?
Or imagine an agent investigating a customer for adverse media. It correctly determines that the customer is not the person mentioned in a criminal investigation.
Good outcome, right?
But the agent reached that conclusion because it mistakenly believed the article was ten years old when it was actually published last week.
Pass?
Or, imagine a servicing agent correctly determines that a customer is eligible for a particular accommodation. In the process, however, it misreads the governing policy, cites an inapplicable exception and accidentally arrives at the correct answer anyway.
The final answer is right. The reasoning is wrong.
Did the system succeed?
I would argue no.
And this points to one of the most important limitations of conventional AI evaluation: A correct answer can still be an evaluation failure.
The three layers of AI agent correctness
Every agent decision can be right or wrong in three ways.
Outcome correctness is the obvious one: Did the agent land on the right answer, approve the right transaction, take the right action? This is where nearly every eval program stops today.
Evidence correctness asks a harder question: did the agent get there on the right facts? Were the sources current and authoritative? Was contradictory evidence addressed instead of ignored? Did the cited evidence actually support the conclusion, or just sound like it did?
Process correctness goes deeper still: did the agent follow a defensible path to get there? Did it perform the required checks before acting? Did it escalate, or hold off, when the evidence wasn’t sufficient?
Most evals only check the first layer. This piece focuses primarily on the other two.
| A deeper dive: The Three Layers of AI Agent Correctness
We have been trained to grade an AI agent’s answer
Most benchmarks are built around a very simple idea.
There is a question. There is a correct answer.
The model produces an answer, and we compare the question and answer.
It matches? Pass.
It doesn’t match? Fail.
For many tasks, that approach is perfectly reasonable.
If I ask a model the capital of France and it answers Paris, I probably do not need a forensic investigation into the cognitive path that led it there. But agents operating inside banks, insurers, healthcare organizations, government agencies and other consequential environments are doing something very different. They are not merely answering trivia questions. Instead, these agents are:
- Gathering evidence
- Resolving identities
- Interpreting documents
- Using tools
- Applying policies
- Making calculations
- Weighing conflicting information
- Taking actions (in certain use cases)
In those settings, the path to the answer becomes part of the answer.
Because the path tells us whether the result is reliable, repeatable and defensible.
Lucky is not the same thing as right in agentic AI evaluations
There is a concept every statistics professor eventually teaches: You can get the right answer for the wrong reason.
Suppose I ask you to predict whether a coin will land heads or tails.
You say heads.
It lands heads.
Were you correct?
Yes.
Did your prediction demonstrate a sophisticated understanding of the physical system?
Probably not. In this case, you were just lucky.
Now, imagine this phenomenon inside an AI agent.
The system makes three mistakes in sequence:
- It retrieves the wrong record.
- It misinterprets the record.
- It applies the wrong rule.
But through some fortunate cancellation of errors, it arrives at the correct final decision.
A benchmark looking only at the final outcome says, Pass.
A serious validator should say, Absolutely not.
Because next time, those mistakes will not necessarily cancel. They may even compound.
The process is inherently unrepeatable, and that’s the difference between measuring an answer and evaluating a system.
In Part 2 of the series, I dive deeper into the three layers of AI agent correctness, and how those can be applied in real-world evaluations.
