-
Real voice prediction result for direct voice input
-
Real voice prediction result for upload file
-
Home Page
Inspiration
Voice cloning has become cheap and convincing. Scammers can now impersonate a family member, a bank officer, or a CEO using just a few seconds of audio. The obvious fix is a deepfake detector, but training a good one usually means collecting huge amounts of real people's voice recordings in one place, which is a serious privacy risk in itself. We wanted to build a defense against fake voices that doesn't require anyone to give up their real one.
What it does
VoxGuard is a deepfake voice detection system built on federated learning. Each client device trains the detection model locally on its own audio and sends only model updates to a central server, never the recordings. The server aggregates those updates into a stronger global model and sends it back. The result is a detector that keeps improving across many users while their voice data never leaves their device.
How we built it
- Data: [dataset, e.g. ASVspoof / WaveFake / your own mix of real and synthetic speech], split across simulated clients to mimic real-world, uneven data
- Features: [e.g. log-mel spectrograms / MFCCs] extracted with [librosa / torchaudio]
- Model: [e.g. a CNN / LCNN / ResNet / RawNet] that classifies audio as real or fake
- Federated training: [Flower / PySyft / custom FedAvg] coordinating local training on each client and aggregating updates on the server
- Evaluation: [metrics, e.g. accuracy, EER, F1] compared against a centralized baseline
Challenges we ran into
- Non-IID data: each client sees different speakers, languages, and kinds of fake audio, which makes the global model harder to converge
- Accuracy vs. privacy: federated training can lag behind centralized training, and we had to tune rounds, learning rates, and aggregation to close the gap
- Generalizing to new deepfakes: detectors that do well on one voice-synthesis method often fail on another
- [Add your own: compute limits, audio preprocessing, communication cost, etc.]
Accomplishments that we're proud of
- Built a working end-to-end federated pipeline for deepfake voice detection
- Showed that detection can be trained without any raw audio leaving a client
- [Add your result, e.g. "reached X% accuracy, within Y points of the centralized baseline"]
What we learned
- How federated averaging works in practice and how data heterogeneity affects it
- How audio features and model choices affect deepfake detection quality
- That privacy and security goals can support each other instead of competing
- [Add anything personal to your team's experience]
What's next for VoxGuard
- Add differential privacy and secure aggregation for stronger guarantees
- Test against newer voice-cloning models and more languages and accents
- Run detection in real time on phones for live call screening
- Deploy on real devices instead of simulated clients