Scam Detector

This is the example of the final output. The final report displays the speech type, confidence. score and risk level.

Inspiration

Scam Detector was inspired by the growing risk of AI-generated voice scams and voice cloning attacks. As AI voice technology becomes more realistic, it is becoming harder for people to recognize whether a phone call is from a real person or a manipulated voice. We wanted to build a system that goes beyond simple accent classification and uses accent patterns, pitch, and frequency-based audio features to detect suspicious voice behavior.

What it does

Scam Detector analyzes an English voice recording and predicts the speaker’s accent pattern as a probability distribution. It also extracts acoustic features such as pitch, MFCC, spectrogram patterns, and frequency changes. By comparing accent probability and acoustic consistency over time, the system estimates whether the voice may be AI-generated, manipulated, or suspicious. Instead of identifying someone’s nationality, Scam Detector focuses on accent similarity and voice stability. If the accent pattern suddenly changes, the pitch becomes unstable, or the spectral features look unnatural, the system can flag the audio as a possible voice scam risk.

How we built it

We built Scam Detector using Python-based audio processing and machine learning tools. First, we collected and prepared English speech data, then converted audio files into useful features such as MFCCs, spectrograms, pitch values, and frequency patterns. These features help the model understand pronunciation, accent, and voice characteristics. We then trained an accent classification model to output accent probability scores. After that, we analyzed how those probabilities changed across different time segments of the same audio clip. This allowed us to create an accent drift feature, which helps detect unnatural changes in speech patterns. Finally, we combined accent probability, pitch stability, spectral features, and model confidence to generate a scam risk score.

Challenges we ran into

One major challenge was making the project more meaningful than a basic accent classifier. Since accent classification already exists, we had to find a stronger real-world application. We decided to use accent and acoustic features as signals for detecting AI-manipulated voice calls. Another challenge was handling the ethical side of the project. We did not want the system to judge or identify someone’s nationality based on their voice. Instead, we designed it to analyze accent similarity and voice consistency only for security and scam detection purposes. We also had to decide which audio features were useful for detecting suspicious voices. A single feature is not enough to judge whether a voice is fake, so we combined multiple signals such as MFCC, pitch, spectrogram patterns, and accent drift.

Accomplishments that we're proud of

We are proud that Scam Detector turns a simple accent classification idea into a more impactful voice security system. Our project does not just show which accent is most likely. It also explains how the voice behaves over time and why it may be suspicious. We are also proud of building an explainable system. Instead of only giving a final result, Scam Detector can show accent probability, pitch changes, frequency patterns, and scam risk. This makes the system easier to understand and more useful in real-world situations such as phone scam prevention.

What we learned

Through this project, we learned how audio data can be processed and analyzed using machine learning. We learned how to convert voice recordings into MFCCs and spectrograms, how accent classification works, and how acoustic features can reveal patterns in human speech. We also learned that AI projects should not only focus on accuracy, but also on real-world impact, fairness, and responsible use. Since voice and accent are sensitive human characteristics, we learned the importance of designing the system carefully so that it helps users without making unfair assumptions.

What's next for Scam Detector

Next, we plan to improve Scam Detector by training it on larger accent and deepfake voice datasets. We also want to compare different models such as SVM, Random Forest, CNN, and Wav2Vec2-based classifiers to find the best-performing approach. We will evaluate the system using metrics such as accuracy, F1-score, AUROC, MAE, and RMSE. We also plan to improve the scam risk score by tuning hyperparameters and testing different audio segment lengths. In the future, Scam Detector could become a real-time phone call warning system that helps users detect suspicious AI-generated voices before they are deceived.