VeriFi

Inspiration

Inspiration Modern financial fraud is no longer limited to just stolen credit card numbers—it is a highly complex, multi-vector attack surface. Hackers bypass standard 2FA, create synthetic identities, hijack active sessions, and sometimes, the threat even comes from inside the bank itself. We realized that relying on a single security checkpoint is an outdated approach. We wanted to build a system based on Defense in Depth—putting up multiple layers of AI-driven security at every stage of a user's interaction with the bank.

What it does VeriFi is an end-to-end, enterprise-grade security ecosystem that protects the entire banking pipeline across 5 distinct phases:

Phase 1: Customer Onboarding (KYC Verification) Before a user can transact, they must prove they are a real, live human. Our Flask web app uses MediaPipe to map facial landmarks in real-time and calculates the Eye Aspect Ratio (EAR) to ensure the user is actively blinking (preventing printed photo bypasses). It then uses OpenCV and DeepFace to compare the live webcam feed against the photo on their uploaded ID card (like an Aadhaar or Driver's License).

Phase 2: Transaction Fraud Detection (The Hard Data) When moving money, a pre-trained scikit-learn pipeline evaluates the "hard data" (amount, time of day, category). It utilizes advanced feature engineering—like calculating the geographic distance between the user's home and the merchant—to calculate a strict fraud probability score.

Phase 3: Behavioral Anomaly Detection (The Soft Data) Sometimes a $20 transaction looks normal, but the person making it is acting strangely. We track "soft data" like session length, click rates, location variance, and device changes. Using a Gaussian Mixture Model (GMM), we score live behavior against historical baselines to detect account hijackings before the transaction even completes.

Phase 4: The Resolution (AI Investigator Agent) Instead of forcing human analysts to dig through database logs, VeriFi automates the investigation. If an anomaly is flagged, the system wakes up Google Gemini (gemini-2.0-flash) via LangChain, feeds it the customer's historical JSON profile, and generates a clean, actionable intelligence report explaining exactly what is suspicious.

Phase 5: Insider Threats (Employee Risk Predictor) The final layer looks inward to catch fraudulent bank employees. A separate dashboard ingests employee metrics (manual security overrides, abnormal working hours) and runs them through a Random Forest Regressor to assign predictive "Risk Scores" and highlight the top riskiest employees.

How we built it We architected VeriFi using a multi-domain AI approach:

Frontend Dashboards: Built using Streamlit and HTML/CSS for both the customer banking view and the internal HR/Employee Risk view. (Note: If you ended up submitting the React/Vite version, make sure to change this bullet point!)

Backend APIs: Powered by Flask to handle the heavy routing and data passing.

Machine Learning: We utilized Scikit-Learn, Pandas, and NumPy to build and train our Random Forest and GMM models.

Generative AI: Integrated Google Gemini and LangChain for our automated Investigator Agent.

Computer Vision: Built our biometric pipeline using OpenCV, MediaPipe, and DeepFace.

Challenges we ran into Merging three entirely different domains of Artificial Intelligence—Traditional ML (Random Forests/GMM), Computer Vision (Liveness/Face Matching), and Generative AI (LLM Analysis)—into one cohesive pipeline was incredibly difficult. Ensuring that the output from the predictive models could be instantly and cleanly parsed by the Gemini LLM required strict data formatting and prompt engineering. Additionally, fine-tuning the Eye Aspect Ratio (EAR) in MediaPipe to accurately detect natural blinking without generating false positives took significant testing.

Accomplishments that we're proud of We successfully built a true "Defense in Depth" ecosystem. We didn't just build a credit card fraud detector; we built a system that ensures the person is physically real, monitors their mouse clicks for hijacking, uses GenAI to explain attacks in plain English, and even polices the bank's own employees.

What's next for VeriFi Our next goal is to implement Graph Neural Networks (GNNs) to map relationships between shared IP addresses and linked devices, allowing the system to catch coordinated, multi-account fraud rings. We also plan to give our Gemini Agent "Agentic Action" capabilities so it can automatically freeze accounts the moment it generates a high-confidence threat report.

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for VeriFi

Built With

• css
• framer-motion-backend-&-apis:-fastapi
• google-gemini-api-machine-learning-&-ai:-scikit-learn
• html-frontend:-react
• javascript
• langchain-computer-vision:-opencv
• languages:-python
• mediapipe
• numpy
• pandas
• recharts
• rest-api
• tailwind-css
• typescript
• vite
• websockets