Inspiration:
A familiar voice can make a suspicious request feel trustworthy. As voice cloning improves and AI agents begin making calls, people need a way to question what they hear without interrupting every legitimate conversation.
CallShield grew from a simple question: could a tiny desktop companion check the audio already playing on your computer and help you decide when to pause and verify a caller? The goal is a useful second signal during a high-stakes conversation, while leaving the decision with the person taking the call.
What it does:
CallShield is a floating desktop pill powered by our Voice Check app. Click to listen to system audio, and its waveform responds to the actual playback. A separate badge to the right updates with Human, AI, or Waiting based on five-second audio windows. Click again to stop. You can drag the pill anywhere and it remembers its position.
A companion browser app lets you upload audio or record your microphone, inspect the synthetic score, and see which part of a clip was analyzed. Analysis runs locally on your computer; audio is not sent to a cloud detection API. The first setup downloads model assets.
The macOS version uses ScreenCaptureKit. The Windows version uses WASAPI loopback and includes tray controls for selecting the correct speaker or headset output. Capture hears the combined playback on its source, rather than isolating a caller. Human and AI are model predictions, not proof of identity; CallShield does not automatically block calls.
How we built it:
The detector uses the existing Forensics 0.3B model, a WavLM and AASIST architecture, through a Python/PyTorch adapter. We did not train the model. We pinned its revision, validated its weights, and preserved its reference preprocessing and score direction.
A FastAPI backend accepts audio only through the local loopback address. FFmpeg decodes it to mono 16 kHz audio. Live mode checks volume and speech activity before inference so quiet windows show Waiting instead of a misleading verdict. File uploads classify the center five seconds; live capture supplies five-second windows.
On macOS, Swift/AppKit hosts the transparent overlay and streams system audio into the local backend. On Windows, C#/.NET and WPF provide the native pill, while NAudio supplies WASAPI capture. The waveform comes from 50 ms RMS measurements before model normalization, so it reacts independently of inference. Capture sessions reject late results and avoid building a backlog of old audio.
Challenges we ran into:
Getting a model to return a label was only the beginning. An early Pella checkpoint contained non-finite weights, so we rejected that path instead of presenting invalid predictions. We also had to handle silence, short clips, mixed system playback, macOS capture permissions, and development signatures that invalidated an earlier permission grant.
The overlay needed careful interaction design: dragging must not accidentally start recording, the verdict must remain readable without dominating the screen, and a smooth waveform must still represent real audio. Windows added output-device routing, DPI-aware positioning, backend startup, and process ownership.
Accomplishments we're proud of:
We built a working local detection pipeline and an unobtrusive desktop interface around it, with upload and microphone testing available separately. Actual macOS system playback produced the expected human and AI labels. The Windows app passed native startup and build checks on a Windows runner, and all 15 Python backend/detector tests passed there. Manual Windows call/headset capture testing is still pending.
We also investigated whether our clone results depended on a watermark. In a matched experiment, all 15 Chatterbox-generated clips were detected both before and after the generator added its watermark. We retained the paired scores, human controls, and simulated phone-channel results rather than selecting only successful examples.
A separate evaluation covered 268 public research clips, with 97.4% correct on the originals and 96.6% on simulated narrowband phone audio. These are fixture results, not live-call accuracy: the data may overlap the model publisher's development/evaluation sources. The Chatterbox experiment covers one generator and five speakers, and does not establish general accuracy on unseen cloning systems.
What we learned:
A detector is most useful when its interface communicates the limits of the evidence. Silence should produce no verdict, a synthetic score should not be presented as confidence, and a human-like result should never replace independent verification of a sensitive request.
We learned to separate audio-capture correctness, model behavior, and interface behavior, and to test each independently. The project also showed how much engineering is needed to turn an existing model into something a person can use during a conversation.
What's next for CallShield:
Next we want to test real calls across codecs, devices, noise levels, and more synthesis engines; improve uncertainty handling; and complete Windows hardware acceptance. We also want signed installers and a clearer path from a warning to independent caller verification.
The current model is published under CC BY-NC 4.0, so commercial deployment would require appropriate licensing. For this prototype, the focus is a transparent, local tool that helps people pause and check before trusting a voice.
Log in or sign up for Devpost to join the conversation.