Intelligence RSS Mesh

Inspiration

RSS feeds revolutionized information distribution in the early 2000s but were abandoned for algorithmic social media. We resurrected RSS technology and enhanced it with modern AI to detect emerging narratives before they trend.

What it does

RSS Intelligence Mesh automatically aggregates news from distributed sources, uses NER (Named Entity Recognition) to extract entities, and detects "information cascades" - when stories gain momentum across multiple outlets simultaneously. GPT-4o generates contextual briefings for each cascade.

How we built it

• Backend: FastAPI + PostgreSQL + Redis + Qdrant vector database
• NLP Pipeline: spaCy for NER, sentence-transformers for embeddings
• AI Synthesis: OpenAI GPT-4o-mini for generating narrative briefings
• Frontend: Next.js with real-time updates
• Deployment: Railway (backend) + Vercel (frontend)

Challenges

• Optimizing Docker images from 8GB to <4GB for Railway's free tier
• Handling Qdrant vector database timeout issues in production
• Balancing aggressive article cleanup with demo data persistence
• Processing 100+ articles in parallel without performance degradation

What we learned

RSS isn't dead - it's a powerful foundation for decentralized information tracking when combined with modern AI. We learned to balance real-time processing with resource constraints and handle distributed system challenges.

What's next

• Fix vector embedding storage for semantic similarity search
• Add sentiment tracking over time
• Build alert system for rapid cascade detection
• Support custom RSS feeds and topics

Built With

• docker
• fastapi
• javascript
• next.js
• openai
• postgresql
• python
• qdrant
• railway
• react
• redis
• sentence-transformers
• spacy
• typescript
• vercel