In this episode of the Model Citizens podcast, FairPlay founder and CEO Kareem Saleh welcomes Ioannis Giannaros, co-founder and CEO, Axle. Axle built Baley, an AI agent that performs entity resolution for sanctions, politically exposed person, and adverse media screening.
They discuss how Baley reviews screening alerts and how that differs from rules-based systems. They also detail FairPlay’s independent evaluation of the agent across more than 1,000 edge cases, and how Axle handles the difference between false positives and false negatives. Giannaros also explained why he recommends keeping a human in the loop, what compliance leaders should ask AI vendors about accuracy testing, and how he expects the compliance analyst role to change over the next three to five years.
Watch the full podcast:
Time Stamps:
[00:00] Welcome and overview
[03:36] Introducing Ioannis Giannaros, Axle, and Baley’s independent evaluation
[05:34] Why screening is more than name matching
[06:45] Building Baley from compliance teams’ standard operating procedures
[09:16] Where AI agents can help with identity resolution, and the limits of rules engines
[10:52] Clearing false positives and taking work off the analyst’s plate
[12:40] Why Axle recommends a human in the loop
[14:06] Why Giannaros and Axle sought independent agent validation
[15:44] What compliance leaders should ask AI vendors
[16:41] The next three to five years in KYC and AML
[19:01] Conclusion
Transcript: Pending validation
Kareem Saleh [00:00]: Welcome back to Model Citizens, the show about how AI is reshaping financial services and what it’ll take to deploy it responsibly. Every day across banks and fintechs, armies of analysts stare at screens, trying to answer a deceptively simple question: Is the Mohammed Hassan who just opened an account that Mohammed Hassan, the person on a sanctions list serving in a foreign government or appearing in last week’s fraud headlines?
This is sanctions, politically exposed person, and adverse media screening. And it sits on the front line of financial crime compliance. And today, much of it is still painfully manual. The lists are enormous. The matches are fuzzy. Names are transliterated, misspelled, hyphenated, shortened, or changed. Dates of birth are missing.
Countries conflict. Legacy rule-based systems generate enormous volumes of alerts, many of which turn out to be false positives. Human analysts then spend their days reviewing records, searching for evidence, comparing identities, documenting their reasoning, and deciding what actually needs to be escalated. In other words, BSA, KYC, and AML may be among the areas of banking most ripe for agentification.
Across financial services, we are moving from AI models that simply flag problems to AI agents that can actually perform work. And financial crime compliance is full of exactly the kind of work agents are good at: high volume, repetitive, evidence-intensive, and expensive. But there’s an obvious catch. These are not low-stakes administrative decisions. A bad call can mean clearing someone who should have been escalated, missing a sanctioned party, or exposing a bank to serious regulatory risk.
So, as AI agents move from assisting compliance professionals to performing work those professionals have done for decades, a much bigger question emerges: How do we know when an AI agent is good enough to trust? My guest today is Ioannis Giannaros, CEO of Axle. Axle has built Baley.
An AI agent that performs entity resolution for sanctions, PEP, and adverse media screening. FairPlay recently put Baley through an independent evaluation. We tested more than 1,000 edge cases and found Baley to be 97.5% accurate. We also examined Baley’s systems architecture, LLM risks, governance, and controls.
Ioannis and I will talk about why financial crime compliance may be one of the first major areas of banking to become truly agentic, where AI agents may outperform traditional rules-based systems, where humans still need to remain in the loop, and what banks should demand before allowing an AI agent to do work historically entrusted to compliance professionals. Ioannis, welcome to Model Citizens.
Ioannis Giannaros: Thank you so much, Kareem.
Kareem Saleh [03:21]: Let’s start with the problem you’re trying to solve. Tell us a little bit about Axle. What is broken about the way sanctions, PEP, adverse media screening works today? And why did you believe this was a problem AI could meaningfully improve?
Ioannis Giannaros [03:36]: Yeah, I appreciate it. So screening, which is what Baley, our agent that we built, accomplishes, is one of those parts of compliance where the work is genuinely ambiguous and is enormous. And then, for the past twenty years, the only answer has been to really just hire more people in this world.
And I saw this in the forefront. I spent my entire career building payment systems, regulated payment systems. So I went out and got regulated all over the world. And as a result of that, we ended up growing a lot of compliance teams as part to manage all of our payment volume that we had. And I saw this firsthand. As growth grew, the only way to scale up was really just to hire more people.
And when we ended up selling that business in 2022, I knew that this was a big problem that I wanted to tackle. And when I dove deeper into the problem, I realized that agents have the exact responsibility and have the knowledge to move away from rules-based engines to move into AI systems that can deduce reasoning and make these judgment calls that they’re exactly built to do. So yeah, it’s one of those problems that AI is really, really good at solving and we’re really excited to spend the past two and a half years building it.
Kareem Saleh [05:12]: That’s awesome. A lot of people hear “screening” and they assume that this is basically sophisticated name matching. Walk us through what Baley is actually doing when it receives an alert from a provider like LexisNexis or ComplyAdvantage. What does the agent have to reason through before it decides that a person should be cleared or escalated?
Ioannis Giannaros [06:45] Yeah, that’s a really good question. And name matching is step zero in the process. The actual work is forming an identity hypothesis, then trying to break it. So what that exactly means and how traditional models used to do this versus what happens today, is that all traditional models are really just rules engines that were built. And there’s some ML around it, but in a rules engine, it simply asks how similar two strings are. And the difference with AI agents is they ask whether the human being is the same person, their customer is the same person on the list. And those are two completely different questions.
And when we start diving into how to build Baley, what we did is we took a lot of standard operating procedure docs from compliance teams. So, this is the operating document of what a human goes through on how they clear an alert. And we simply built, step by step, the process and the SOP of what an agent should go through.
Now, the first step for this was name matching and in traditional sense, it used to be all rules-based processes and you’ll take a name and the similarity, but it’s so much more than this. Understanding paternal, cultural naming structures, understanding the naming conventions in different countries, that matters a lot. And that’s simply something that a rules engine cannot process. So sending this over to a reasoning engine is something that it’s really good at.
Now there’s a lot more steps in what Baley does as well. And it’s similar steps to what the compliance analysts are doing today. They’ll look at, in the structured data that you get back from LexisNexis or ComplyAdvantage or any other screening data providers, you’ll look at, is there a date of birth on there? And if it’s not there in the structured data, what a compliance analyst would do is they’ll go into the articles, build a knowledge graph around who the customer or who the sanctioned person is and try to identify, okay, did this person hold a position at a certain time period? And does that give us an indication of the age range of when our customer would have been at that same time period?
So there’s a lot of different cues that you can use in addition to doing photo identification matching that compliance analysts go through that we’ve built into an AI agent. And with that, after building all these workflows, we’re able to clear a lot of this work that’s currently being escalated as false positive, which ultimately means in higher costs for the bank because they have to spend labor to actually clear these queues and take that off of analysts’ plate and only focus on cases that really need to be looked at.
Kareem Saleh [08:50]: That’s so interesting. Now, my understanding is that one of the hardest parts of screening is identity resolution, right? Transliterated names, aliases, changed surnames, incomplete dates of birth, different countries, conflicting information. Where do you think agents have an advantage over traditional rules based systems in dealing with that kind of ambiguity?
Ioannis Giannaros [09:16]: Yeah, good question. This ties a little bit more into the answer I just mentioned as well. Rules-based systems are once again taking two strings and trying to find similarities between the two strings. And then with AI agents, you’re able to build knowledge graphs around that data and put different data pieces together to draw conclusions and build a hypothesis, very similar to what analysts are doing today.
And then going back to, there’s different naming conventions, there’s additional information that you can get from data, such as naming conventions, whether it’s a Spanish maternal name, you can draw assumptions on, hey, maybe we didn’t get a gender for this sanctioned person, but we understand that this name is 98% a female name and we’re able to draw conclusions that our male customer is not the same person as this person on the list.
So screening is and evaluating false positives within screening is one of those use cases that is perfect for AI agents.
Kareem Saleh [10:23]: Now, FairPlay tested Baley across more than a thousand different edge cases and found 97.5% accuracy. But as you know, in compliance, the mistakes can matter just as much as the overall accuracy number. How do you think about the trade-off between false positives that waste analysts’ time and false negatives that can expose a bank to real regulatory risk?
Ioannis Giannaros [10:52]: Yeah, this is a really good question as well. It’s something that we think about a lot and also what stemmed us trying to get this independent evaluation from FairPlay and really excited about that we did it. The important thing is that these two things are not symmetric and should never be treated as symmetric. False positives increase the costs of the bank and then a false negative increase costs in relation to a consent order.
So we never treat these as the same, but with every customer that we onboard, we always try to understand what is that balance for every organization.? And people are on different spectrums on this. In the sense that some people have a higher risk appetite, some people have a lower risk appetite, and we try to assess, hey, where is that? And we tested with FairPlay, we tested over a thousand different cases. Twenty-six of those cases we got incorrect, which we acknowledge. And the way that we manage that with an organization is by building confidence scores and then asking the question, “Hey, does that number need to be zero?” And doing a tuning exercise during the pilot period with the customer.
And then understanding that, hey, we need three pieces of evidence to actually clear out this case. We need two pieces of evidence. It could be one piece of evidence. And that relays back to a confidence score that we can then adjust to a customer’s risk appetite. So it’s something that we do during the pilot phase with customers and something that we, before onboarding, going live with our systems, it’s something that we spend a lot of time on.
[12:40] The second thing that we always do with customers in relation to making sure that we have accuracy, recall targets, precision, whatever the important metric is for the customer, is we always recommend just having a human in the loop, right? A lot of these cases, it’s never a recommendation. Even though we feel very confident for a lot of L1 and perhaps even L2, it could be designed in a way where you don’t need a human in the loop.
But our recommendation is always, let’s run this with a human in loop, review the cases. And even with that, there’s a lot of ROI in the pilot or whether going live with the customer because all the work is done, all that the human has to do is just press “approve” or “reject” on the case and escalate it for further review. And either way you go, that is our recommendation, having a human in a loop, which is always a good safeguard to have in these AI systems.
Kareem Saleh [13:41]: Now, you voluntarily put Baley through our evaluation that looked not only at performance but also at architecture and LLM risks, governance and change management. Could you say a little bit about why independent validation was important to you? And what do you think banks should demand from vendors before allowing an AI agent to perform work historically done by compliance professionals?
Ioannis Giannaros [14:06]: Yeah, another great question. I don’t think any vendor, any regulator has used the word “trust us” as a control, right? And you need to back up your models with data and more specifically, you need to back up your models with independent data. And I mean, these are simple model risk management principles that have been in machine learning environments for years.
And we thought that it was important for us to take the same steps in our AI agents, and we’re gonna continue to do it. It’s not something that we do once a year. It’s not something we do, set it and forget it. These models change and we have to consistently evaluate these models on a yearly, monthly, quarterly basis. And to be frank, it’s something that all the large models like Anthropic, OpenAI, they consistently benchmark their models. And it’s something that’s very public and a lot of software vendors are very cagey about it in the sense that, our metrics are our metrics. We’re not gonna show anyone where we’re benchmarking, but we just wanted to take a more open approach and we’re really excited to encourage a lot of other software vendors to come out with their accuracy for their models and to be spoken about what they’re producing internally. It’s good work and there’s a lot of great work that’s happening in the space and I think people should be very confident in displaying that work and educating compliance leaders what’s important to see in these metrics as well.
[15:44] I think for compliance leaders in banks, they should be asking their vendors on their accuracy metrics or precision, recall, all the standard ML metrics, I think it’s very, very important to ask for. But I think what’s more important is who measured it. Is it something that they’re running internally? What’s the time frame? Are they doing it monthly? Is it every time they release a new update?
I think understanding the governance around model testing is very, very important. And I think every single bank and compliance leaders should be aware of asking other vendors on that. They should be asking their vendors those questions for sure.
Kareem Saleh [16:18]: If we were to look three to five years ahead, how much of today’s KYC and financial crime compliance workflow do you think will be performed by AI agents? And where do you think the humans will remain essential, whether because of judgment, regulation, accountability, or something else?
Ioannis Giannaros [16:41]: Yeah, this is a good question. I think about this a lot, especially with the future of Axle as well. Like, where do we see ourselves playing and how do we design these systems in a future AI world? I think the analyst’s job is moving from processing to supervision and judgment. And I think we’re starting to see that in engineering pretty heavily, where top-tier engineers right now, especially in our company and across the board, are using AI systems right now to manage to manage coding, actual coding, right?
And I think it’s gonna be the same thing with compliance analysts. And I think the role of an analyst in the future is gonna be much more technical in the sense that they’re gonna need to understand these AI systems a lot more. And it’s a problem that’s not going away. And I think with the rise of synthetic fraud and a lot of synthetic, AI-driven KYC onboardings, this is a problem that’s gonna only expediate in the ecosystem, and the only way to tackle that is with AI systems that can really tackle these patterns of fraud that are very sophisticated across the entire bank organization.
So it’s a problem that’s not going away. It’s a problem that’s gonna get more sophisticated. And the only way to tackle it is really to use AI systems. There’s no going around it. We’re past this point of no return on AI systems. And in addition to that, with it growing, you can’t just scale headcount to manage these false positives, and I think it’s something that banks need to really take a look at and get ahead of in the sense that they need to make sure that the software vendors are able to handle this and it’s something that I’m very excited about as well.
And yeah, that’s where I think the future of KYC and AML systems are heading, much more sophisticated AI systems that are built into the bank’s processes and the role of the analyst is gonna move from processing to supervision and judgment.
Kareem Saleh [19:01]: Ioannis, thanks for taking time today to share your thoughts on what it’ll take to responsibly deploy AI agents in high-stakes workflows like BSA, KYC, AML. We’re great admirers of what you’re building at Axle and grateful for the partnership.
Ioannis Giannaros: Awesome. Thank you so much, Kareem.
