FakeNews?

A multimodal AI vision/reasoning tool that watches a video alongside you and detects, corrects, and explains false claims in real-time.

Comment

🛡️ FakeNews?

Real-time multimodal AI for truth detection.

💡 Inspiration

In a world saturated with deepfakes, deceptive advertising, and viral misinformation, determining the truth has become a full-time job. Researching sources manually is exhausting, and the consequences of "fake news" are real—ranging from financial fraud to physical violence.

We built FakeNews? to bridge this gap, providing a real-time shield that detects and corrects false claims as you watch them.

⚙️ What it does & How We Built it

We developed FakeNews? as a Google Chrome extension, ensuring it works seamlessly across YouTube, Facebook, TikTok, Instagram, and major news outlets like CNN or Fox News. Our analysis pipeline operates in three sophisticated layers:

👤 Speaker Identification: Uses facial detection to identify who is speaking and attribute claims to the correct person.

📝 Claim Extraction: Employs speech-to-text transcription and NLP to pull verifiable statements from the audio.

🔍 Multimodal Verification: Uses the Gemini API via our Custom Search API to perform live Google Search queries.

Instead of a simple "True/False" label, we provide contextual verification, highlighting specific claims and providing links to supporting evidence so users understand why a statement is misleading.

🚧 Challenges we ran into

📍 Usability & Impact: We didn't want to build a tool that stayed in a vacuum. We spent an hour conducting market research on NC State’s campus, interviewing students and mentors to ensure our tool actually improved upon their current fact-checking workflows.

🎯 Validation Accuracy: Avoiding "false accusations" was a priority.

🖼️ The Solution: We implemented a system that captures a frame of the content you are watching including the speaker. This image is sent to Gemini, which uses facial recognition to sync the speaker's identity with the transcribed audio for pinpoint accuracy.

🏆 Accomplishments that we're proud of

🚀 Rapid Development: Successfully built a working multimodal AI Chrome extension from scratch in less than 24 hours.

📱 Cross-Platform Versatility: Created a pipeline that handles video content across diverse platforms like YouTube and TikTok with ease.

🤝 Speaker Attribution: Solved the complex problem of linking facial recognition with transcription to reduce misattribution.

🧠 Explainable AI: Developed a system that doesn't just flag content but explains the reasoning behind the fact-check, empowering the user.

Built With

Share this project:

Updates