Inspiration
Deepfakes drive misinformation and harm, threatening online safety. Deepfake Sentinel empowers users to detect manipulated images in real-time, protecting digital trust. Built for SunHacks 2025, it targets Best Hack for Safety (mitigating deepfake risks) and Best AI Agent (intelligent multi-agent analysis).
What it does
Upload Image: Via Streamlit UI. Face Detection: MTCNN highlights faces (>90% confidence). Deepfake Classification: Hugging Face model labels “real” or “fake” with confidence and risk levels. AI Agent Analysis: CrewAI (Analyst, Coordinator, Assessor, Reasoner) or direct Mistral reasoning explains results (e.g., “Fake due to texture artifacts”). Output: Visuals + insights in a clean dashboard.
How we built it
UI: Streamlit Face Detection: MTCNN, OpenCV, PIL Classification: Hugging Face Transformers (dima806/deepfake_vs_real_image_detection), PyTorch AI Agents/Reasoning: CrewAI, LangChain-HuggingFace, Mistral-7B-Instruct-v0.1/v0.2 Deployment: Ngrok for public demos Testing: Using Roboflow testing dataset found at (https://universe.roboflow.com/asd-w743e/deepfake-xrppf/images/02c0FTmZFNnWgZKWKIcG)
Challenges we ran into
Model Selection: Tested multiple Hugging Face models; settled on dima806 for accuracy. Switched from Llama-3 to Mistral-7B to bypass API rate limits and improve reasoning stability. Deployment: Faced Streamlit/ngrok issues with API stability and public URLs. Resolved with environment-based tokens and robust error handling.
Accomplishments that we're proud of
Built two versions: CrewAI with face detection and streamlined classifier with Mistral reasoning. Achieved >90% classification accuracy with clear reasoning (e.g., “Fake due to lighting mismatches”). Ensured demo reliability with fallbacks.
What we learned
Through building this Deepfake Detection App, I gained practical experience in combining computer vision models with LLMs to create a usable application. I learned how to:
Deploy pretrained models: I used Hugging Face’s deepfake classification model and understood how to preprocess images with AutoImageProcessor and generate predictions using AutoModelForImageClassification.
Integrate reasoning with LLMs: I added a reasoning step using a text generation model (like Mistral) to explain why an image might be classified as real or fake. This helped me see how LLMs can enhance interpretability of ML predictions.
Streamlit application building: I developed an interactive interface where users can upload images, get classification results, and receive confidence scores with reasoning, improving accessibility of ML models.
Handling challenges: I worked through issues such as model selection (VL vs text-only), Hugging Face Inference API integration, and formatting inputs/outputs properly for different models.
Overall, this project taught me how to bridge image classification (vision models) with explainability (language models) and present the results in a user-friendly way. It strengthened my skills in model deployment, multimodal reasoning, and app development.
What's next for Deepfake Sentinel: AI-Powered Safety Shield
• Add video deepfake detection. • Enhance agent collaboration for real-time alerts on platforms like X. • Optimize model for faster inference on edge devices.
Log in or sign up for Devpost to join the conversation.