iSpy | Devpost

Upload_Page
Main_Page
Case_Page
Image_Page
Case_List_Page

🚀 Project Summary: Multi-Modal Insurance Fraud Detection

The conception of this project was largely influenced by one of the team members' personal experiences with being scammed, which led to the research on the damages of scams and fraud in the insurance world. This excursion led to the concretizing of a software solution that will ameliorate the significant financial losses incurred by insurance companies annually due to fraudulent claims, ultimately protecting the billions of financial capital of insurance companies.

💡 Core Challenge & Solution

| Solution | Developed a multi-model system that fuses unique results from three different models: Claim-Level Classifiers, AI-Generated Image Detectors, and Fraud Fixture Detectors. |

🛠 Key Technical Accomplishments

Weighted Ensemble Model: Built, tuned, and combined the outputs of three models into a weighted ensemble.
Performance: Achieved approximately 80% precision on claims flagged as fraudulent.
Data Imbalance Handling: Successfully managed the challenge of highly imbalanced data (fraud constituted only $\approx 6\%$ of the dataset) through careful sampling and threshold adjustments.

🌱 Challenges & Lessons Learned

| Core Challenge | Combining tabular claim data with vehicle images into a single, unified fraud detection pipeline. |

Final Outcome: Created a robust, multi-model fraud detection system and learned how to directly connect Machine Learning outputs to tangible real business impact.

🔗 Gemini API Integration

It parsed insurance claim PDFs into a standardized JSON schema with dozens of key fields, enabling consistent analysis across different insurer formats. It transformed combined risk scores from images and documents into structured JSON reports, delivering key findings, risk factors, and actionable recommendations.

By enforcing strict JSON outputs and applying robust error handling, Gemini allowed the system to unify document understanding with risk interpretation, completing the multi-modal fraud detection workflow.