AI Repo Analyzer

Autonomous multi-agent platform performing AST complexity parsing, static security scanning, and sandboxed pytest execution to automate repository due diligence and technical screening.

Comment

Inspiration

Evaluating open-source repositories and conducting technical due diligence manually takes hours of tedious code reading. Furthermore, relying solely on standard LLMs for code reviews often results in generic advice or hallucinated line references.

We were inspired to build AI Repo Analyzer to bridge the gap between deterministic software engineering tools and probabilistic AI reasoning. By combining static AST analysis with autonomous multi-agent orchestration, we created a system that delivers deep, zero-hallucination repository audits in seconds.

What it does

AI Repo Analyzer is an autonomous, multi-agent engine designed for technical due diligence, codebase auditing, and automated portfolio evaluation.

  • Deterministic Static Analysis: Parses codebases using Radon for AST complexity metrics, identifies code smells, and executes static security scans.
  • Sandboxed Execution: Clones code into isolated environments to run unit tests (pytest) and verify runtime health.
  • Anti-Hallucination Loop: Cross-verifies LLM reasoning against actual static metrics to guarantee line-level accuracy.
  • Automated Deliverables: Generates executive health scores, auto-renders interactive Mermaid.js architecture blueprints, produces line-level refactoring roadmaps, and posts automated PR reviews via Qodo.

How we built it

  • Agentic Orchestration: Structured a multi-agent system (Manager agent coordinating static, security, and test sub-agents) using the TrueForge agent harness.
  • Core Intelligence: Powered by Gemini and Groq API endpoints for high-speed diagnostic synthesis and architectural reasoning.
  • Real-time Streaming: Built an asynchronous backend utilizing FastAPI and WebSockets to stream diagnostic outputs instantly to the user interface.
  • Integrations: Connected to GitHub API and Exa MCP (Model Context Protocol) servers for repository context fetching and deep web diagnostics.

Challenges we ran into

  • Eliminating AI Hallucinations: Standard LLMs frequently invent line numbers or misinterpret repository structures. We solved this by creating a verification loop that forces the model to validate its claims against raw Radon AST parsed data and sandboxed execution logs before writing reports.
  • Multi-Agent State Synchronization: Managing real-time WebSocket state updates across multiple sub-agents executing concurrent diagnostic tasks required designing robust asynchronous event streaming patterns.

Accomplishments that we're proud of

  • 50%+ Audit Acceleration: Reduced full technical due diligence and code screening time by over half while maintaining high accuracy.
  • Production-Grade Architecture: Built a complete pipeline that combines static AST analysis, sandboxed execution, real-time WebSocket streaming, and dynamic Mermaid.js blueprint generation.
  • Autonomous Reliability: Successfully implemented strict guardrails ensuring every refactoring recommendation is backed by line-level proof.

What we learned

  • Hybrid AI Design: Discovered that LLMs perform exponentially better when paired with deterministic tools (like Radon AST parsers) rather than operating in isolation.
  • Agent Orchestration: Gained deep experience managing multi-agent workflows, state persistence, and low-latency streaming through Model Context Protocol (MCP) servers and agent harnesses.

What's next for AI Repo Analyzer

  • CI/CD Integration: Developing a native GitHub Action to run automated agentic code audits on every pull request automatically.
  • Automated Hotfix Generation: Extending agents to not only suggest refactoring roadmaps, but automatically open pull requests with verified code fixes.

Built With

Share this project:

Updates

Submission history