TrustAgent AI

A real-time AI agent that detects psychological manipulation and voice scams during live calls.

Comment

💡 Inspiration

Every 15 seconds, a vulnerable individual—often a senior citizen—loses their hard-earned savings to Vishing (Voice Phishing) and social engineering scams. Current solutions are reactive; they help you after the money is gone. I wanted to build a proactive, Zero-Trust shield. An OS-level agent that intercepts, analyzes, and neutralizes threats in real-time before the OTP is ever shared.

⚙️ What it does

TrustAgent OS is a context-aware, real-time cyber telemetry dashboard.

  • Live Interception: It securely records incoming audio and transcribes it instantly.
  • Contextual AI Analysis: It doesn't just look for keywords; it analyzes the psychological intent of the caller to detect Identity Theft, Extortion, or Tech Support scams.
  • Dynamic Mitigation Playbook: If a threat is detected, it generates exact steps based on what the user leaked (e.g., "You mentioned SBI. Immediately block your card").
  • Regional Cyber-Cell Routing: It uses reverse geocoding to lock the user's coordinates and dynamically generates a Google Maps route to their nearest local cyber police station, along with national helpline numbers.

🛠️ How we built it

We architected the system for maximum speed and zero bloat:

  • Frontend: Built with vanilla HTML/CSS/JS for an ultra-lightweight, sleek, CISO-style dashboard. No heavy frameworks, ensuring millisecond UI updates.
  • Backend: Powered by a FastAPI Python server hosted on Render.
  • AI Core: Integrated the Groq API for blistering fast inference. We used Whisper-large-v3 for Speech-to-Text and engineered strict JSON-schema prompts for Llama-3.3-70b-versatile to handle the heavy contextual and psychological analysis.
  • Geolocation: Integrated BigDataCloud API for reverse geocoding.

🚧 Challenges we ran into

Building a real-time audio pipeline is notoriously difficult. We faced several major roadblocks:

  1. Cross-Browser Audio Codecs: Different devices (Android vs. Desktop) record in different raw formats (MP4 vs. WebM) while stripping headers. This caused the AI to reject the files as "corrupted." We built a dynamic codec resolver in JS that formats the Blob perfectly before transmission.
  2. AI Hallucinations (The Silence Trap): When the mic was open but the room was silent, the Whisper model hallucinated phrases like "Ignore static or silence." The LLM flagged this as a threat! We engineered a strict "Noise-Gate" filter in Python to catch ambient noise and force a 0% Threat baseline.
  3. Mid-Hackathon Model Deprecation: Literally hours before submission, Groq decommissioned the llama-3.1 model we were using, causing a sudden API 400 Error. We had to rapidly migrate to llama-3.3-70b-versatile and update our fail-safe UI shields to ensure the dashboard wouldn't crash during the transition.

🏆 Accomplishments that we're proud of

  • Achieving sub-3-second end-to-end latency from voice recording to generating a dynamic mitigation playbook.
  • Building the Zero-Trust Memory Shield on the frontend, ensuring that network jitters or API errors never wipe the user's screen but gracefully handle the exception.
  • Creating a truly global tool that adapts its emergency contact info based on the user's live GPS coordinates.

📚 What we learned

  • The intricacies of the browser MediaRecorder API and raw audio chunking.
  • Advanced Prompt Engineering to force LLMs to output strict, robust JSON structures without markdown wrapping.
  • Building fault-tolerant UI components that don't break when asynchronous data fails.

🚀 What's next for TrustAgent OS

  • Telecom Integration: Directly integrating the agent with telecom provider APIs to automatically sever the call if the Threat Level exceeds 90%.
  • Mobile Native App: Porting the web OS into a background Android/iOS service that runs seamlessly during active phone calls.
  • Multi-lingual Support: Allowing the AI to detect localized scams in regional languages like Hindi, Tamil, and Bengali.

Built With

Share this project:

Updates