-
-
Web Interface Verify Mode
-
Web Interface Gather Mode
Inspiration
During the COVID-19 pandemic, misinformation spread rapidly across social media through text messages, shared URLs, and images. False claims about cures, vaccines, and preventive measures often influenced public health decisions and reduced trust in scientific institutions. Many existing misinformation detection systems rely on keyword-based filtering or text-only machine learning models, which fail to capture contextual meaning or detect multimedia misinformation. This motivated us to develop a system that can verify health claims in multiple formats while remaining transparent, explainable, and privacy-preserving.
What it does
The project is a multi-modal COVID-19 misinformation detection system that verifies health-related claims submitted as text, URLs, or images. The system analyzes the claim and compares it with verified information from trusted sources such as WHO, CDC, and NHS. Based on the analysis, it classifies the claim into three categories:
- CORRECT
- MISINFORMATION
- UNVERIFIED The system also provides confidence scores, supporting evidence, and explanations, helping users understand why a claim was classified in a particular way.
How we built it
The system was developed using Python with a modular architecture and integrates several AI and data processing technologies. The web interface was built using Streamlit, allowing users to submit claims through text, URLs, or image uploads. Image-based claims are processed using EasyOCR to extract text, while langdetect and deep-translator enable multilingual input support. For semantic understanding, the system uses Sentence-BERT (all-MiniLM-L6-v2) to generate contextual embeddings of claims. These embeddings are compared with entries in a locally maintained knowledge base using cosine similarity. To detect repeated misinformation patterns, the system implements MinHash and Locality Sensitive Hashing (LSH) to build a similarity-based claim network. spaCy Named Entity Recognition is used to extract entities such as vaccine names and organizations for validation against structured data. A local Gemma 2B language model running through Ollama generates explanations for classification results. All components are integrated into a decision engine that combines semantic similarity, entity validation, intent detection, and network risk scoring to produce the final result.
Challenges we ran into
One major challenge was designing a system capable of detecting misinformation across multiple input formats, including text, images, and web pages. Extracting reliable text from images and ensuring consistent preprocessing across different input types required careful integration of OCR and text cleaning techniques. Another challenge was balancing accuracy and explainability. Many high-performing models are black-box deep learning systems, but we wanted a solution that could provide transparent and understandable decisions. Additionally, ensuring that the system runs efficiently on standard hardware without relying heavily on external cloud services required optimizing the architecture and caching embeddings.
Accomplishments that we're proud of
We successfully developed a multi-modal misinformation detection framework capable of analyzing claims from different sources in a unified pipeline. The system combines semantic AI models, rule-based reasoning, and network-based similarity analysis to provide reliable verification. Another key achievement is the offline and privacy-preserving architecture, which reduces dependence on external APIs while maintaining transparency and reproducibility. The system also produces explainable outputs with confidence scores and supporting evidence, helping users understand how decisions are made.
What we learned
Through this project, we gained hands-on experience in Natural Language Processing, semantic embeddings, and similarity search techniques. We learned how models such as Sentence-BERT can capture contextual meaning and detect paraphrased misinformation. We also explored network analysis algorithms like MinHash and LSH, which allow efficient detection of near-duplicate claims. Additionally, we learned how to design modular AI systems that integrate multiple components such as preprocessing pipelines, knowledge bases, and decision engines while maintaining explainability and scalability.
Explanation of How the Project Uses AI
Artificial Intelligence is used in the system through semantic language models and similarity-based analysis. The core AI component is Sentence-BERT, which converts textual claims into semantic embeddings that capture contextual meaning rather than relying on simple keyword matching. These embeddings allow the system to compare user claims with verified knowledge using cosine similarity, enabling detection of paraphrased or semantically similar misinformation. The system also uses spaCy Named Entity Recognition to identify entities such as vaccines, drugs, and organizations within claims. Network-based algorithms such as MinHash and Locality Sensitive Hashing (LSH) help detect repeated misinformation patterns by identifying claims that are similar to previously known misinformation. Additionally, a local large language model (Gemma 2B) generates explanations for classification results, improving interpretability without replacing rule-based verification logic.
What Makes the Project Unique
The project stands out because it combines multi-modal input processing, semantic AI models, and network-based analysis in a single framework. Unlike many existing systems that analyze only textual misinformation, this system can verify claims from text, URLs, and images. Another unique aspect is the integration of semantic similarity with claim similarity networks, enabling detection of paraphrased misinformation and repeated misinformation patterns. The system also emphasizes explainability and transparency, combining rule-based validation, entity verification, and AI-generated explanations. Additionally, the system operates offline with locally stored knowledge bases and embeddings, ensuring privacy and reducing dependence on external services.
What's next for COVID-19 Misinformation Detection: A Network Approach
Future work will focus on extending the system beyond COVID-19 misinformation to cover broader public health and medical misinformation topics. We also plan to integrate real-time social media monitoring and support additional multimedia formats such as videos. Further improvements include expanding the knowledge base with automated fact-checking pipelines, improving multilingual support, and deploying the system as a scalable web platform that can assist healthcare organizations and policymakers in combating misinformation during future public health crises.
