Pipeline
Multi-Modal Visual Forensics Engine
Smart Coverage Decoding System
Precision Loss Valuation Model
RAG-Augmented Compliance Agent

🛡️ SwampGuard: Demystifying Insurance Claims with Multi-Modal AI

Originally built at SwampHacks 2026

💡 Inspiration: Reality in the Sunshine State

In Gainesville, "The Swamp" is more than just a nickname for the football team—it is the perfect metaphor for the reality of navigating the insurance claim process. We live in a state where insurance premiums are skyrocketing, carriers are fleeing, and every annual hurricane season brings a fresh wave of anxiety.

The inspiration for SwampGuard came from seeing our community struggle against this bureaucracy. We realized the core problem is Information Asymmetry. Insurance adjusters have enterprise tools and legal teams; homeowners have a PDF policy they can't understand and a smartphone full of photos.

We asked: Can we build an AI-powered "Public Adjuster" that empowers the user? We wanted to turn a chaotic pile of photos into a professional, Xactimate-style estimate that stands up to scrutiny.

💻 What it does

SwampGuard is a full-stack intelligence platform that audits property damage claims before they are submitted.

Multi-Modal Vision Analysis: Users upload multiple photos of damage. We use a hybrid AI pipeline to analyze the severity of the damage (e.g., determining if drywall needs simple drying vs. complete replacement).
Dynamic Cost Estimation Logic: This is the core engine. The AI determines the nature of the repair from images, while the user provides the scale (approximate square footage).
- The Equation: $$\text{Est. Cost} = f(\text{AI Severity Index}) \times \text{User Area Input} \times \text{Xactimate Unit Price}$$
- The more photos the user provides, the more accurate the severity assessment becomes, leading to a tighter price range.
Legal & Policy Compliance: The chatbot is RAG-augmented with Florida Statutes (627.70131) to answer questions like "How long does the insurer have to pay me?" with legal accuracy.

⚙️ How we built it

We prioritized a cutting-edge hybrid architecture to balance performance and intelligence.

The Backend (FastAPI + Hybrid AI)

Our backend is a high-performance FastAPI application that orchestrates two distinct AI models:

The Gatekeeper (Local OpenAI CLIP): To optimize for latency, we deployed a local CLIP model. It acts as a zero-shot classifier, instantly filtering irrelevant images and tagging damage types on the server side before they hit the cloud.
The Analyst (Gemini 2.0 Flash): We pipe the filtered images into Gemini’s massive context window. Gemini acts as the "Adjuster," assigning a severity score (1-5) to the damage. This score maps to specific repair codes (e.g., WTR DRY vs WTR RPLC), which are then multiplied by the user's input area to generate the final estimate.

The Frontend (React 19)

We built the UI with React 19, leveraging new Hooks for cleaner state management.

Styling: We used TailwindCSS 4 for a utility-first, flat design.
Visualization: Recharts renders the claim history, while jsPDF allows users to export a professional dossier.

🚧 Challenges: The Hybrid Architecture

Our journey wasn't a straight line; integrating local and cloud AI models created unique hurdles.

The "Latency vs. Intelligence" Trade-off: We initially considered a pure cloud approach, but realized that sending high-res images to an LLM for simple classification was too slow.

The Solution: We chose to deploy CLIP locally. This created a "Dependency Hell" on our Windows development environment, where transformers and PyTorch versions clashed, causing silent backend crashes. We spent hours pinning dependencies to get the local inference engine running smoothly alongside the async FastAPI routes.

The Result: A system that gives instantaneous feedback on image uploads (via CLIP) while still delivering deep, multimodal insights (via Gemini).

🧠 What we learned

Data Density = Accuracy: We learned that a single photo is often ambiguous. By encouraging users to upload multiple angles, the AI's "understanding" of the room's geometry improves significantly.
Hybrid AI is Powerful: Combining a small, fast local model (CLIP) with a large, smart cloud model (Gemini) is the sweet spot for real-world applications.

🚀 What's next for SwampGuard

Data-Driven Iteration: Theoretically, accuracy scales with data volume. We plan to fine-tune our models on large-scale enterprise datasets of historical claims to improve the precision of the "Image-to-Severity" mapping.
Enterprise Integration: Connecting our JSON output directly to Xactimate's API to inject line items directly into the insurer's software.
Mobile App: Porting to React Native for offline-capable documentation immediately after a storm.