Extension pop-up screen
Extension pop-up settings
Verifying bank legitimacy
Medium risk warning with score
Active defense label
High fraud risk with score and threat summary

Inspiration

In 2025, over 57% of adults worldwide were targeted by scams, leading to a staggering $442 billion in global losses. This financial crisis is the result of a single major shift in fraud: attackers are no longer just hacking systems, they are hacking people. Social engineering has become so advanced that it easily bypasses almost every traditional security layer a bank has in place.

Today, attackers use AI to create a sense of total legitimacy. They can generate professional websites that look identical to real banks and use AI voice deepfakes to impersonate trusted officials over the phone. These methods are so convincing that even cautious people are manipulated into voluntarily handing over their sensitive information. Because the user provides this information willingly, traditional banking defenses usually do not catch the fraud until it is already too late.

Hence, we created SherLock, a multimodal agentic system that shifts the focus to the early attack chain by identifying social engineering threats before a victim ever enters their data.

What It Does

SherLock is a proactive, cross-device fraud prevention ecosystem consisting of a Chrome Extension and a companion mobile app. It acts as an autonomous digital guardian that calculates a real-time Trust Score for every interaction.

We built two components:

Mobile App

Detects suspicious phone calls by analyzing voice signals and scam-related language patterns. If a scam is detected, it sends a warning signal to the user’s browser.

Chrome Extension

The extension automatically triggers whenever a user visits a site with input fields like usernames or passwords. It follows a multi-step verification process:

Domain Verification: Compares the URL against dynamic whitelists and blacklists.
Content Extraction: If a domain is unrecognized, SherLock strips the HTML content and uses an LLM to summarize the site’s intent.
Contextual Analysis: It compares the site's content against the URL to determine if the requested information is warranted. This prevents "look-alike" domains from tricking the user.

The system assigns a risk level:

Safe – website appears legitimate
Warning – suspicious signals detected
Dangerous – blocks credential entry to prevent scams

How We Built It

Our system combines a mobile AI detection app and a Chrome extension that communicate with each other.

The mobile app analyzes phone conversations for scam indicators, while the extension analyzes websites in real time. When signals from both sources indicate high risk, the extension escalates protection and prevents users from entering sensitive information.

Challenges

Our biggest challenge was minimizing false positives. Many legitimate sites ask for sensitive data, so simply flagging every login page was not an option. We solved this by using the LLM to verify if the "ask" matched the "identity" of the site. Another hurdle was synchronizing the audio analysis from the phone with the browser extension with low enough latency to prevent a user from typing while still on the call.

What We Learned

We learned that effective security must be as multimodal as the attacks themselves. Fraud isn't just a "web" problem or a "phone" problem; it is a cross-platform psychological attack. Building SherLock taught us how to leverage agentic AI to step in exactly when human judgment is most likely to be compromised.

Future Improvements

With more development time, we would expand the system to include:

Expanding the agentic monitoring to flag suspicious links within Gmail or Outlook.
Advanced Audio Biometrics: Improving the deepfake detection to identify even the most sophisticated voice clones.
Automated Reporting: Automatically drafting and sending fraud reports to the relevant financial institutions the moment a high-risk scam is blocked.

Our long-term goal is to create a personal AI security assistant that protects users from digital scams across all devices.