🚀 Fraud Detector with AI

🌟 Inspiration

Fraud detection has always been a cat-and-mouse game. Attackers are becoming more sophisticated, and traditional rule-based systems can’t always keep up. I was inspired by the idea of combining speech-to-text, prompt engineering, and fraud detection models to build an interactive system that flags fraud in real time.

The goal was to let users interact naturally (via speech or text) while still detecting suspicious patterns with high accuracy.

💡 What It Does

Our system is an AI-powered fraud detector that:

Takes voice input from the user.
Converts it to text using OpenAI Whisper.
Applies fraud detection prompts to classify the input as LEGIT ✅ or FRAUD ❌.
Gives results back via text and speech (using pyttsx3).
Provides a clean React frontend for user interaction.

It’s like having an AI fraud analyst that listens and responds instantly.

🛠️ How We Built It

Backend (Python + Flask)

Integrated openai-whisper for transcription.
Used structured prompt engineering for fraud detection.
Implemented pyttsx3 for text-to-speech feedback.

Frontend (React)

Built a simple UI for interaction.
Users can record speech, view classification, and listen to results.

📚 What We Learned

Prompt engineering is powerful but requires iteration—specific wording dramatically changes results.
Deploying ML systems is as much about dependencies and infra as it is about models.
React + Flask integration taught us about API communication pitfalls.

⚡ Challenges We Faced

Prompt Tuning: Model gave vague answers until we enforced structure:

  You are a fraud detection system. Classify the following transaction as FRAUD or LEGIT.

🔢 Math Behind It

To evaluate fraud probability, we experimented with a simple risk formula:

$$ P(\text{fraud}) = \frac{\text{risk score}}{\text{total score}} $$

This concept helped explain confidence levels in decisions.

🚀 Accomplishments We’re Proud Of

Built a working pipeline from speech → fraud classification → speech output.
Created an intuitive UI for demo purposes.
Learned end-to-end integration of multiple AI tools.

🔮 What’s Next

Expand fraud models to include transaction data, anomaly detection, and user behavior.
Add a dashboard for analytics & reporting.
Deploy as a cloud-based microservice for scalability.
Train on domain-specific fraud datasets for higher accuracy.