--- title: "The Three Layers of AI Agent Correctness" url: "https://fairplay.ai/the-three-layers-of-ai-agent-correctness/" description: "Most evaluations treat AI agent correctness as a single question. It's actually three questions, covering the outcome, the evidence, and the process." author: "Kareem Saleh" date_published: "2026-10-02" date_modified: "2026-10-02" image: "https://fairplay.ai/wp-content/uploads/2026/10/AI-Agent-Correctness.png" --- > More FairPlay resources: fetch the curated index at https://fairplay.ai/llms.txt # The Three Layers of AI Agent Correctness Most evaluations treat AI agent correctness as a single question: did the agent get the right answer? It’s actually three questions, covering the outcome, the evidence the agent relied on and the process it followed to get there. An agent can pass the first and fail the other two, and a test that only checks the final answer will score that as a success. In [my previous post in this three-part series](https://fairplay.ai/how-an-ai-agents-correct-answer-can-be-an-evaluation-failure/), I covered how an AI agent’s correct answer can be an evaluation failure when you measure solely at the answer level rather than evaluating AI systems as a whole. Today, we’ll dive into the three layers of agent correctness and how they can be used to perform a more holistic evaluation. Now, consider an adverse-media agent whose task is to determine whether a news article describing alleged fraud relates to a particular customer. The agent concludes: “This article does not concern the customer.” That conclusion happens to be correct. But suppose the customer is Michael Johnson, born 1978, Chicago. The person in the article is Michael Johnson, born 1982, Chicago. The agent explains that they are different people because they live in different cities, except for the fact that they do not. Both live in Chicago, so in this case, the agent invented a distinction. It reached the correct answer because some other fact (perhaps the dates of birth, for example) separates the individuals. If your evaluation only checks the final classification, this is a successful test. But what did we actually learn? We learned that the agent reached the right answer while relying on a falsehood as evidence. That should worry us more because now we know that the system has a process capable of manufacturing **persuasive but unsupported evidence**. The *outcome* concealed the *defect*. ## How the three layers of agent correctness can help We need to start thinking about agent correctness in layers. ![Three Layers of AI Agent Correctness](https://fairplay.ai/wp-content/uploads/2026/10/Three-Layers-of-AI-Agent-Correctness-1024x640.png) The first layer is the obvious one: **outcome correctness.** - Did the agent reach the right answer? - Did it approve the right transaction? - Identify the right customer? - Apply the correct classification? - Take the appropriate action? Necessary, but insufficient. The second layer is **evidence correctness.** Did the agent rely on information that was accurate, relevant and sufficiently supported? - Did it identify the correct documents? - Use the correct customer record? - Distinguish authoritative sources from weak ones? - Recognize stale evidence? - Separate corroborating evidence from merely interesting evidence? - Avoid fabricating facts? That is a different question entirely. An agent may reach the correct answer while failing the evidence test. In a high-stakes environment, that should often count as a failure. ## The third layer of AI agent correctness: Process correctness There is a third layer. If all of the evidence is accurate, the agent still might apply it incorrectly. Imagine an insurance agent evaluating a claim. It correctly identifies the following: - the policy - the relevant coverage provision - the date of loss - the claimed event - and the applicable exclusion So far, so good. But then it applies the exclusion before checking whether an exception to the exclusion applies. The exception does apply. Luckily, the agent later notices another provision that leads it back to the correct coverage determination. Again: Correct answer. Correct documents. Broken process. That matters because the next case may not contain the second provision that accidentally rescues the decision. Process correctness asks: - Did the agent perform the necessary steps? - Did it perform them in a defensible sequence? - Did it satisfy required conditions before acting? - Did it verify critical information? - Did it handle contradictions appropriately? - Did it escalate when required? - Did it respect the boundaries of its authority? - Did it use tools in the intended way? This determines the difference between **“The answer happened to be right,”** and, **“The system behaved correctly.”** Those are not synonymous. ## Think about how we validate humans We already understand this intuitively in other fields. If a student solves a math problem and writes: 2 + 2 = 5 5 – 1 = 4 Therefore the answer is 4. You do not give full credit. Why? The final answer is correct, but the method is not. If an auditor reaches the correct financial conclusion using the wrong supporting records, we do not say, “No harm, no foul.” If a doctor reaches the correct diagnosis because she misread an X-ray but happened to guess correctly based on an unrelated symptom, we would not describe that as an exemplary diagnostic process. If a bank declines an applicant for the correct ultimate reason but sends an adverse action notice listing a different and inaccurate reason, the fact that the lending outcome was correct does not magically make the explanation acceptable. In fact, that exposes a major risk to the bank. In consequential systems, **how you got there matters**, and AI agents should not get a special exemption. ## The danger of post hoc coherence There is an additional wrinkle with generative AI. Agents are exceptionally good at producing explanations, but that can create a dangerous illusion. A system takes an action. You ask why. It gives you a beautifully structured explanation with three supporting facts. 1. A policy citation. 2. A logical progression. 3. A confident conclusion. It *looks* like an audit trail, but the explanation may not accurately reflect what actually drove the outcome. Or, some of the cited evidence may be irrelevant. Perhaps the reasoning may have been reconstructed after the fact. A polished explanation is not necessarily a faithful one. This means agentic assurance cannot simply ask, **“Can the system explain its answer?”** The better questions are: **“Does the explanation correspond to the evidence actually used?”** and **“Was that evidence appropriate?”** and **“Would changing that evidence change the outcome in the way we would expect?”** Those are much harder tests, but they’re also much more meaningful. ![Slide contrasting polished explanations with actual evidence used](https://fairplay.ai/wp-content/uploads/2026/10/AI-Agent-Post-Hoc-Coherence-1024x640.png) ## Counterfactuals can expose the wrong reason One of the most powerful ways to determine whether an agent’s output is tied to the right reasoning is to **perturb the evidence**. Suppose an agent approves an application. You believe it should have relied primarily on income, debt obligations and credit history. **Now, change a fact that should be irrelevant,** *perhaps the applicant’s first name***.** Does the outcome change? Change formatting in the document. Does the outcome change? Move an irrelevant paragraph to the top. Does the outcome change? **Now, change something that should matter.** *Reduce verified income substantially, for instance.* Does the outcome remain unchanged? *Increase a material obligation.* Does the decision respond? *Remove the document that supposedly supported the conclusion.* Does the agent still produce exactly the same answer? These kinds of tests tell us something that ordinary accuracy cannot; namely, whether the system is **sensitive to the right things and insensitive to the wrong things**. That is much closer to validating reasoning. *In Part 3 of this series, we’ll explore how agentic validation can help determine whether the AI agent was right for the right reason.*