Bluff Busters

Inspiration

Phone scams have become one of the most common ways people lose their life savings, often targeting seniors, immigrants, and less tech-savvy users. Existing solutions usually try to block numbers or detect keywords, but scammers constantly adapt, leaving people vulnerable once the call has already started. We wanted to create a solution that focuses on the conversation itself—helping users stay aware, in control, and empowered in real time.

What it does

Bluff Busters is a real-time scam call detection app that listens to conversations (with user consent), transcribes speech into text, and uses AI to detect manipulative patterns such as urgency, fear, and impersonation. Instead of just blocking numbers, it analyzes how scammers actually behave. Users see a live transcript along with a clear risk level—Low, Medium, or High—and the reasons behind it. This allows them to recognize manipulation and act confidently before sharing sensitive information, protecting themselves and their families.

How we built it

We built Bluff Busters as a full-stack web application with a clear separation of services. The frontend uses HTML and JavaScript to capture microphone input, display the live transcript, and show risk levels. The backend is powered by FastAPI, handling asynchronous streaming to ensure smooth real-time performance. A rolling 30-second context of the conversation is maintained and fed into an AI model that detects scam behavior and provides explanations. Services for transcription, context management, and AI analysis are modular, making the system efficient, scalable, and ready for real-world use.

Challenges we ran into

One of the biggest challenges was achieving real-time performance while keeping transcription and AI analysis accurate. Managing conversation context so that the AI could detect scams without slowing down the system required careful design. We also faced difficulties handling varied speech patterns, including accents, interruptions, and slang, while keeping latency low. Designing a user interface that was intuitive, clear, and non-alarming for vulnerable users was another hurdle. Finally, balancing speed with predictive accuracy required multiple iterations and testing.

Accomplishments that we're proud of

We successfully built a complete end-to-end system that goes beyond a demo, providing live AI-powered scam detection with actionable feedback. The interface clearly communicates risk levels and explanations, giving users control rather than simply issuing warnings. The backend is modular and efficient, capable of handling real-time streaming, transcription, and AI analysis smoothly. Overall, we demonstrated a responsible use of AI to reduce both financial and emotional harm in real-world scenarios.

What we learned

Through this project, we learned how to integrate speech-to-text and AI analysis in a real-time pipeline using asynchronous design. We realized that keeping users aware and empowered is as important as technical accuracy. Building a full-stack system taught us the importance of clear service separation and efficient data flow. We also learned how to handle natural conversation in AI pipelines with robust preprocessing and context management, and the value of iterative testing for real-time applications.

What's next for Bluff Busters

Looking forward, we plan to expand our AI models to detect more types of scams and subtler manipulation tactics. Improving transcription accuracy for diverse accents and languages is another priority, along with adding mobile support to protect users on smartphones. We also aim to incorporate adaptive learning so the AI can improve from user feedback, and explore partnerships with financial institutions and senior care organizations to scale the impact of Bluff Busters to more vulnerable users.

Built With

• css
• elevenlabs
• fastapi
• gemini-api
• html
• javascript
• python
• speech-to-text
• uvicorn