Mirmer AI

Resurrecting 400 years of peer review tradition - where AI models debate, critique, and synthesize answers at machine speed.

Comment

Inspiration

In 1665, the Royal Society established peer review as the foundation of scientific truth. For 400 years, expert panels have deliberated, debated, and synthesized knowledge - but always constrained by human speed and geography. What if we could resurrect this time-tested process and run it at the speed of AI?

The "which AI should I trust?" problem inspired Mirmer. GPT-4 gives one answer, Claude gives another, Gemini suggests something different. Instead of choosing, we asked: what if AI models could peer-review each other? What if we brought back the expert council - the very system that built modern science - and turbocharged it for the age of artificial intelligence?

What it does

Mirmer AI resurrects the traditional expert council process through a 3-stage AI deliberation system:

Stage 1: Independent Review - Multiple AI models (GPT-4, Claude, Gemini, Llama, etc.) respond to your query simultaneously and independently, just like experts submitting their initial assessments.

Stage 2: Anonymous Peer Review - Each model anonymously evaluates and ranks the other responses, identifying strengths, weaknesses, and insights - mirroring blind peer review that ensures objectivity.

Stage 3: Chairman Synthesis - A designated model synthesizes the collective wisdom into a comprehensive answer, weighing the peer rankings and extracting consensus.

What took weeks in academic committees now happens in seconds. The ancient process lives again, at machine speed.

How we built it

We resurrected peer review with modern technology:

Backend (Python + FastAPI)

  • Async/await architecture for true parallel model queries via OpenRouter API
  • Server-Sent Events (SSE) for real-time streaming of the deliberation process
  • Dual-mode storage: PostgreSQL for production, JSON for local development
  • Firebase authentication for secure access
  • Razorpay integration for subscription management

Frontend (React + Vite)

  • Tabbed interface showing each stage of the council process in real-time
  • Real-time streaming updates as models deliberate
  • Conversation management with search functionality
  • Clean, responsive UI built with Tailwind CSS

Python SDK & CLI

  • Official mirmerai package for programmatic access
  • Browser-based authentication flow for CLI tools
  • Async and sync client implementations
  • Full conversation management capabilities

Infrastructure

  • Backend deployed on Railway with PostgreSQL
  • Frontend deployed on Vercel
  • Published SDK on PyPI for easy installation

Challenges we ran into

1. Making competitors collaborate - AI models are designed to compete, not cooperate. Creating a system where GPT-4 fairly evaluates Claude's response (and vice versa) required careful prompt engineering to ensure objectivity.

2. Real-time streaming complexity - Orchestrating 3 stages with multiple models while streaming updates in real-time was architecturally challenging. We needed SSE for streaming, proper error handling for model failures, and state management across stages.

3. CLI authentication paradox - How do you implement browser-based OAuth (Firebase) for a command-line tool? We solved this by creating a popup-based flow with a local callback server - bridging two incompatible paradigms.

4. Cost optimization - Running multiple premium models per query is expensive. We implemented smart caching, usage tracking, and tiered pricing to make it sustainable.

5. Anonymous peer review at scale - Ensuring models couldn't identify each other's responses while maintaining context was tricky. We strip identifying markers and randomize presentation order.

Accomplishments that we're proud of

Digitally resurrecting a 400-year-old process - We didn't just build another AI wrapper; we brought back the deliberative process that built modern civilization.

True parallel execution - All models query simultaneously using async/await, making the council process blazingly fast.

Blind peer review that works - Models genuinely evaluate each other objectively, producing rankings that reveal consensus and outliers.

Real-time deliberation - Users watch the council process unfold live, seeing how AI models debate and synthesize knowledge.

🛠️ Production-ready architecture - Dual-mode storage, proper authentication, payment integration, SDK, CLI tool, and comprehensive documentation.

📦 Published Python SDK - Available on PyPI as mirmer, making the council process accessible to developers everywhere.

What we learned

Ancient wisdom scales - The peer review process isn't obsolete; it's timeless. When applied to AI, it solves the trust problem better than any single model can.

Async is essential - Without true parallel execution, the council process would be too slow. Python's async/await made the resurrection possible.

Streaming changes everything - Watching the deliberation unfold in real-time creates trust. Users see the process, not just the result.

AI needs checks and balances - Just like human experts, AI models benefit from peer review. The synthesis is consistently better than any individual response.

Developer experience matters - Building a great SDK and CLI tool turned Mirmer from a web app into a platform.

What's next for Mirmer AI

🎓 Academic partnerships - Integrate with research institutions to apply the council process to scientific literature review and hypothesis evaluation.

Enterprise features - Custom model selection, private model hosting, and team collaboration features for organizations.

Specialized councils - Domain-specific expert panels (medical, legal, technical) with models fine-tuned for specific fields.

Transparency metrics - Show consensus scores, disagreement analysis, and confidence levels to help users understand the deliberation quality.

API marketplace - Let developers create custom council configurations and share them with the community.

Research mode - Allow researchers to study how different model combinations perform on various types of questions.

The peer review process took 400 years to perfect. We've resurrected it for the AI age - and we're just getting started.

Built With

Share this project:

Updates