iOS: Detailed safety information for a number
iOS: Look up phone numbers
iOS: Turning phone calls to text for fraud detection
iOS: Live keyword analysis of caller for spam
Android: Detailed safety information for a number
Android: Fraud score shown for all incoming calls
Android: Look up phone numbers
Phone fraud is an illegitimate $23,000,000/year industry that hurts friends and family so we decided to make iOS and Android apps to help stop this issue.
What it does
The product consists of both Android and iOS apps connected to our database. The app uploads data about unsolicited phone calls for every user and assigns fraud scores based on weighted metrics. This data is searchable through the app and shown each time a number calls the user (on Android). Along with this, the IBM Watson speech to text API is used to listen to phone calls for keywords (on iOS) and alert users that the call is illegitimate and they may be in the process of getting scammed.
How it works
We built a custom back-end using MySQL and PHP with native apps running on both iOS and Android. Upon first-time startup, the app takes a user's call log and queries all unsolicited calls to our database where its logged for further analysis (on Android). A fraud score is assigned for all numbers in the database based on how frequently a number has contacted multiple people, user submitted reports, and whitelisted numbers (scraped from multiple API's, such as Capital-One's Nessie API). The AI call analysis feature works by analyzing live call recordings for keywords that are prevalent with common scams. When the AI finds a two or more keyword match, the user is alerted and given the option to drop the call.
Challenges I ran into
The iOS platform is very limiting in how much call related information can be accessed, so we had to work around this by implementing as many features as we could on this platform. For example, iOS doesn't allow apps to record live phone calls so we made the app simulate another phone and join incoming calls as a conference call to get audio output. However, we couldn't reliably get the audio output, so instead we switched to a simpler speaker based approach.On the Android side of things, the most difficult part was detecting when a phone call was being recieved. Unfortunately, we weren't able to implement voice recognition in the Android version of the app either, due to time constraints.
Accomplishments that I'm proud of
This was one of the first projects we've made integrating phone call detection features, and we were able to get it to detect spam calls with a very reasonable amount of accuracy. Along with this we were able to make something that can hopefully save people from getting scammed.
What I learned
On the Android side of things, this was certainly a useful learning experience for how to listen for calls and send notifications to the user.
What's next for FraudSource - Smart Spam Call Detection
Improving the weighting for our fraud score metrics would be the top priority, along with adding more metrics and improving the speech to text accuracy of the AI.