-
-
here_banner
-
architecture_flow
-
dashboard_mockup
Inspiration
Most lending systems use a single model to approve or reject loan applications. If that model has a blind spot, there is nobody to catch it. I wanted to build something that mimics how banks are actually supposed to do model risk management, where an independent team challenges every decision the main model makes. That framework exists in real banking regulation (SR 11-7) but almost nobody builds it at the software level because it is expensive and complicated. Agents make it tractable.
What it does
FriskFleet runs every loan application through two completely independent AI agents and makes them argue about it.
The first agent (Gemini 2.5 Flash) scores the applicant by pulling income data, credit bureau records, and uploaded documents through three MCP tool servers. The second agent (Gemma 3 on Vertex AI) reviews the exact same application without seeing the first agent's answer, then comes to its own conclusion. When they disagree, a third component generates a plain-English explanation of why their reasoning diverged.
A drift monitor runs on a daily cron job and tracks whether the two models are drifting apart over time, flagging when their agreement rate starts to slip.
The dashboard shows live decisions, divergence analysis, batch underwriting across multiple applicants at once, and lets you export a full compliance audit trail as JSON.
How we built it
The architecture follows the Three Lines of Defense model used in real bank model risk management.
Line 1 is the Scoring Agent: Gemini 2.5 Flash with three MCP servers for income verification, credit bureau data, and document storage, all running on Cloud Run.
Line 2 is the Challenger Agent: Gemma 3 on Vertex AI with a completely separate scoring pipeline, also on Cloud Run.
Line 3 is the Drift Monitor: a Cloud Run service triggered by Cloud Scheduler that tracks statistical agreement between the two models and writes results to Firestore.
The frontend is a Next.js app deployed on Cloud Run. All infrastructure is provisioned with Terraform. Service-to-service auth uses dedicated GCP service accounts with minimum required IAM roles. API keys are stored in Secret Manager.
Challenges we ran into
The Gemma 3 endpoint on Vertex AI has stricter input formatting requirements than Gemini. For a while the challenger agent was silently falling back to Gemini Flash and agreeing with everything, which made the drift rate look perfect but was completely meaningless. Catching that took longer than expected because the fallback was silent.
Getting two models to disagree in a genuinely useful way also took tuning. If they agree too much the whole system is pointless. If they disagree on everything it is noise. The goal was calibrated disagreement.
Wiring seven Cloud Run services together with proper IAM, VPC ingress rules, and Secret Manager access was more involved than it looks from the outside.
Accomplishments that we're proud of
The divergence analysis actually produces useful output. When Gemini approves someone that Gemma rejects, the explanation card tells you concretely which factors drove the gap, not just "the models disagreed."
The whole infrastructure tears down and rebuilds from a single Terraform apply. Every service account has exactly the permissions it needs and nothing more.
The drift monitor running on a real Cloud Scheduler cron job means the system keeps working and self-reporting after you walk away from it.
What we learned
How much complexity hides inside a multi-agent system once you start caring about isolation. It is easy to build two agents that share state and call each other. It is harder to build two agents that are genuinely independent so their disagreements actually mean something.
Also learned how real bank model risk management works at the regulatory level. The SR 11-7 framework is surprisingly well-thought-out and maps almost directly onto what a multi-agent architecture naturally produces.
What's next for FriskFleet
Connecting to a real credit bureau API instead of the mock MCP server. Adding a human-in-the-loop review queue for cases where the two agents disagree above a certain confidence threshold. Expanding the drift monitor to track individual feature distributions, not just overall agreement rate. And running a proper backtesting pipeline against historical loan data to see whether the challenger actually catches errors the champion misses.
Log in or sign up for Devpost to join the conversation.