FiscalSentry

Autonomous financial defense agent powered by Gemini 3.7 & Genkit that intercepts bills, audits statutory overcharges, and executes real dispute workflows across Google Workspace & Slack.

Comment

Inspiration

Every month, millions of consumers and businesses lose hard-earned capital to administrative billing friction. From surprise out-of-network hospital charges violating the Federal No Surprises Act (45 C.F.R. § 149) to unbundled medical procedure codes, hidden SaaS price hikes, and duplicate vendor line items, modern billing systems are complex and intentionally opaque.

When an unexpected \$1,800 hospital bill or a surprise renewal arrives, resolving it requires hours of tedious labor: cross-referencing insurance policies, calculating fair-market percentiles, citing federal regulations, drafting formal appeal letters, and staging calendar deadlines. Most people lack the time, legal knowledge, or energy to fight back.

We asked ourselves: Why should financial self-defense require manual labor?

Rather than building another passive chatbot that waits for user prompts and generates text, we set out to build FiscalSentry — a true autonomous Taskmaster agent that works 24/7 in the background, intercepts messy financial documents, audits statutory violations, and executes multi-step dispute workflows across enterprise tools with zero human friction.

What it does

FiscalSentry is an autonomous financial defense engine powered by Gemini 3.7 Flash, Firebase Genkit, and Google Cloud. It replaces passive conversational loops with proactive, real-world execution:

1. 24/7 Autonomous Background Sentry

Operating via Google Cloud Scheduler and Firebase Cloud Functions (2nd Gen / Cloud Run), FiscalSentry autonomously polls connected Gmail feeds and inbound webhooks. It filters out promotional marketing noise and extracts financial statements (PDFs, invoices, receipts) without requiring user intervention.

2. Multimodal Statutory Auditing with Gemini 3.7 Flash

FiscalSentry feeds messy, multi-page documents into Gemini 3.7 Flash to perform deep line-item auditing with hybrid reasoning:

  • Statutory Violations: Detects illegal balance billing under the No Surprises Act, unbundled procedure codes under CMS NCCI guidelines, and hidden renewal markups.
  • Algorithmic Recovery Calculation: Evaluates total recoverable delta \( \Delta \text{Savings} \) across all \( n \) line items: $$ \Delta \text{Savings} = \sum_{i=1}^{n} \max\left(0, \, \text{Billed}_i - \text{Benchmark}_i\right) $$
  • Multi-Currency Normalization: Reconciles transactions across USD (\$), INR (₹), EUR (€), and GBP (£).

3. Multi-Channel Autonomous Execution DAG (Directed Acyclic Graph)

When overcharges or statutory violations are detected, FiscalSentry stages and executes real-world actions:

  • Google Calendar: Automatically schedules 30-day statutory appeal deadlines.
  • Google Tasks: Creates prioritized action items complete with phone numbers and negotiation call scripts.
  • Google Sheets: Reconciles audited line items into real-time financial tracking spreadsheets.
  • Slack & Discord: Dispatches rich alert cards to financial operations channels.
  • Vector PDF Generator: Produces signature-ready legal appeal dossiers with official statutory citations and CFPB debt collection notices (12 C.F.R. § 1006).

4. Zero-Knowledge Client-Side Cryptography

User financial data is encrypted with AES-256-GCM client-side before synchronization to Cloud Firestore, backed by a 14-day rotating cryptographic epoch to ensure total privacy and data sovereignty.

How we built it

FiscalSentry is built on a modern, decoupled architecture combining Google Agent frameworks and Google Cloud infrastructure:

  • AI & Agent Core:
    • Gemini 3.7 Flash accessed via @google/genai for high-speed multimodal OCR and advanced legal/statutory reasoning.
    • Firebase Genkit (@genkit-ai/core, genkit) for structured prompt flows, schema typing, and tool calling.
    • Google Antigravity Agent Spec for autonomous subagent delegation.
  • Google Cloud Infrastructure:
    • Firebase Cloud Functions (2nd Gen / Cloud Run): Serverless event triggers and secure API endpoints.
    • Google Cloud Scheduler: Scheduled cron daemon for hourly automated inbox surveillance.
    • Cloud Firestore: Encrypted, real-time database for financial audits and action states.
    • Firebase Hosting: Edge-optimized CDN hosting for the global web application.
  • Frontend Workstation:
    • Next.js 14 (App Router), Tailwind CSS, Framer Motion, and Lucide Icons.
    • Custom vector-based PDF rendering engine ensuring 100% crisp typography without external font-encoding glitches.
    • Shimmering skeleton loading system eliminating cold-start zero-value flashes during authentication.

Challenges we ran into

  1. Autonomous OAuth Token Lifecycles in Background Crons: Google OAuth access tokens expire every 60 minutes. Running an autonomous background worker on Cloud Scheduler required building a silent token refresh rotation mechanism that securely updates tokens in Firestore without user interaction.
  2. Messy Multi-Page Tabular Document Ingestion: Real-world medical bills and vendor quotes often have irregular table layouts and poor scan quality. We solved this by leveraging Gemini 3.7 Flash’s frontier multimodal capabilities paired with strict Genkit schema validation to output structured, validated JSON every time.
  3. PDF Unicode Font Encoding Artifacts: Standard PDF 8-bit Helvetica fonts corrupted UTF-8 emojis into garbled binary glyphs (Ø=þáp). We re-architected our PDF generation pipeline to draw pure vector geometric primitives (drawBrandMark) directly on the PDF canvas, ensuring pixel-perfect branding and multi-page text wrapping.
  4. Hydration Flickers During Client Decryption: Because audits are decrypted client-side with AES-256-GCM, the dashboard initially flashed \$0.00 before Firestore records hydrated. We engineered an isInitialLoading state and pulse skeleton loaders to ensure a seamless, flicker-free user experience.

Accomplishments that we're proud of

  • True Autonomous Execution: Built an agent that doesn't just answer questions, but proactively intercepts paperwork, audits legal violations, and executes actions across external APIs.
  • Zero-Knowledge Privacy: Built an architecture where sensitive financial and medical documents are encrypted on the client before reaching cloud databases.
  • Multi-Currency & Multi-Statute Intelligence: Successfully reconciled cross-border currencies (USD, INR, EUR, GBP) and complex regulatory frameworks (No Surprises Act, CMS NCCI, CFPB).
  • Production-Ready & Fully Deployed: A live, fast, and responsive web application deployed to Google Cloud / Firebase Hosting with sub-second response times.

What we learned

  • Agentic Workflows Require Robust Fallbacks: Autonomous background agents need idempotent action dispatching and deterministic error recovery (e.g., handling rate limits or offline webhooks gracefully).
  • Gemini 3.7 Flash’s Reasoning Power: Gemini 3.7 Flash's hybrid thinking and tabular comprehension make it exceptionally accurate at parsing complex medical coding charts (CPT/HCPCS/NCCI) without hallucinations.
  • Decoupled Architecture Scales Faster: Separating the ingestion layer (Cloud Functions/Scheduler), intelligence layer (Genkit/Gemini), and presentation layer (Next.js/Firebase Hosting) made debugging and extending features seamless.

What's next for FiscalSentry

  • Autonomous Voice Negotiation with Gemini Live: Integrating real-time bidirectional audio to allow FiscalSentry to place dispute calls directly to billing departments and navigate IVR phone trees.
  • Open Banking & Plaid Direct Sync: Connecting live bank accounts and credit cards to automatically trigger dispute workflows the moment an erroneous charge posts.
  • Enterprise Multi-Tenant Compliance Suite: Adding inline Model Armor guardrails, team collaboration workspaces, and automated accounting ERP sync (QuickBooks, NetSuite, SAP).

Built With

Share this project:

Updates