Multi-Agent Open-Source Project Analysis Assistant

A nested multi-agent system powered by Gemini 3, using ReWOO planning for task decomposition and knowledge integration, with dynamic tooling and persistent memory to boost research efficiency.

Comment

Inspiration

We were inspired by the growing complexity of open-source projects and the need for intelligent analysis tools. Researchers and developers spend countless hours manually reviewing documentation, code, and community discussions. We envisioned a system that could automate this analysis process using Gemini 3's advanced reasoning capabilities combined with a multi-agent architecture to provide comprehensive insights into open-source projects.

What it does

The Multi-Agent Open-Source Project Analysis Assistant is an intelligent research platform that automatically analyzes open-source projects across multiple dimensions. It uses a nested multi-agent architecture powered by Gemini 3 to collect research data, analyze documentation and code, and synthesize comprehensive reports. The system integrates with GitHub through GitMCP, performs web searches via Tavily, accesses Wikipedia knowledge, and includes multimodal processing capabilities for analyzing diagrams and screenshots. All interactions are stored in Mem0-based memory for continuous learning.

How we built it

We built this system using NVIDIA NeMo Agent Toolkit as the foundation framework. The architecture features a main reasoning agent that delegates to specialized sub-agents: research_collector for data gathering, content_analyzer for document processing, and knowledge_synthesizer for final analysis. We implemented custom multimodal processing functions and integrated multiple data sources through function groups. The entire system is configured through YAML files and orchestrated using LangChain agents with Gemini 3 as the core reasoning engine.

Challenges we ran into

The primary challenge was the fundamental architectural limitation of Router Agent in NeMo Agent Toolkit - it only accepts string inputs while intelligent agents (ReWOO, ReAct) require ChatRequestOrMessage format. This incompatibility prevented direct routing to agent branches, forcing us to either modify the core framework code or create wrapper functions. We ultimately redesigned the architecture using a nested reasoning pattern with Reasoning Agents to maintain clean separation of concerns while achieving the desired multi-agent collaboration.

Accomplishments that we're proud of

We're proud of:

  • Successfully implementing a working multi-agent reasoning architecture without modifying the core NeMo Agent Toolkit codebase
  • Creating a custom multimodal processing function group that handles images, documents, and code visualization
  • Integrating multiple disparate data sources (GitMCP, web search, Wikipedia, memory) into a cohesive system
  • Developing a nested reasoning pattern where each agent layer specializes in specific tasks while maintaining compatibility
  • Building a system that can generate comprehensive research reports automatically

What we learned

Through this project, we learned:

  • The importance of understanding the input/output contracts between different agent types in multi-agent systems
  • How to effectively use function groups to organize related capabilities
  • The power of nested reasoning agents for complex task decomposition
  • Practical experience with Gemini 3's reasoning capabilities and how to leverage them effectively
  • Techniques for handling network reliability issues when integrating external services

What's next for Multi-Agent Open-Source Project Analysis Assistant

Future enhancements include:

  • Adding more specialized agents for specific analysis types (security analysis, performance benchmarking, license compliance)
  • Implementing interactive visualization of analysis results
  • Expanding multimodal capabilities to support video and audio analysis
  • Creating a plugin architecture for community-contributed analysis modules
  • Developing a web-based interface for non-technical users
  • Adding support for more version control systems beyond Git
  • Implementing collaborative analysis features where multiple users can contribute to the same analysis

Built With

  • a2a-(agent-to-agent)-protocol
  • docker
  • fastapi
  • glm-4.7
  • google-gemini-3
  • langchain/langgraph
  • langfuse
  • mcp-(model-context-protocol)
  • mem0
  • next.js
  • node.js
  • npm
  • nvidia-nemo-agent-toolkit
  • nvidia-nims
  • nvidia-rag
  • openai-api
  • opentelemetry
  • phoenix
  • pydantic
  • python-3.11+
  • ragaai-catalyst
  • redis
  • tavily-search-api
  • uv
  • weave
  • websockets
  • wikipedia-api
Share this project:

Updates