ScamShield

Inspiration

One day my uncle who just got fired from a job received a scam call from a person who said the he would give him a job if he paid him $1000. And my uncle who found this as a golden opportunity fell for it and gave the scammer $1000 (which was his life savings). The scammer never called back or attended any of my uncle's calls. My uncle lost his $1000 and fell into a lot of depression. This is what inspired me and my teammates to create ScamShield an AI powered App that protects people from scams.

What it does

Our product, ScamShield, utilizes the power of AI to transcribe calls and analyze them to determine the scam patterns in the call. The way that this works is first, the user upload either an audio file of the call, or a transcript of the call. If the file submitted was an audio file, then the app converts it to text using sherpa.onnx. After that is done, the transcription is analyzed by a free tier model of Gemini API. This model looks at what the potential scammer is saying and checks for common scammer tactics as well as other identifiers of scam calls. After that is done, the app will display a risk rating which a percentage that indicates how likely it is that the call was a scam call. Underneath, it displays 5 reasons on why the AI thinks this is a scam. This helps people protect themselves in the future. In order to fully verify the scam the app gives three smart questions to ask the scammer and expose him. The answers to these questions by the scammer is analysed by the AI to completely verify the scam.By doing all of this, the app is able to accurately identify scams and protect people from becoming the victims of scams.

How we built it

We used android studio for both the frontend and backend. We also used two different AI models, Sherpa.onnx for converting audio to text, and Gemini for determining the risk.

Challenges we ran into

It was hard working together at the same time for two reasons. The first is that Android studio doesn't natively support live updates so two people could work on the app at the same time. And second, the varying time zones also made it hard to find times when everyone could work together. We tried running a local scam analysing model instead of the free Gemini API to ensure privacy and security, but we weren't able to do that because we didn't have the resources to fine tune our own AI model and optimize it during the 7 day period. Despite these problems, we persevered and in the end, our team was able to create a really good final product.

Accomplishments that we're proud of

We are proud that we build a fully functional app that analyzes scam patterns and outputs clear follow-up actions that the user should take in just 7 days. ( after pulling a lot of all nighters)

What we learned

We learned a lot about training AI models, building apps, how you should never give up when a lot of problems arise, teamwork and we gained a lot of valuable experience and knowledge for the future.

What's next for ScamShield

We tried fine-tuning a model called Qwen2.5-3B however this model is currently too large to run on a mobile phone. In the future we want to optimize it and make it so that we have a more precise lightweight model that runs entirely on your mobile phone to ensure privacy and security. We also want to continue to develop this app so that it looks and feels more professional. After that, we hope that we can publish this app and make a real impact in people's lives.