AgentDesk

Inspiration We wanted to tackle the Nashik HackDays problem statement #2: "Autonomous Multi-Agent Workflow Platform." Most multi-agent frameworks operate purely in the terminal, making the interaction feel like a black box to end-users. We were inspired to build a platform that brings these invisible agent handoffs into the light. We wanted a gorgeous, Claude-inspired UI where you could literally watch the agents "think," share context, execute code, and compile reports in real-time.

What it does AgentDesk is a desktop application that acts as an autonomous multi-agent document analysis team. You can drop in a PDF, a raw text file, or paste a GitHub repository URL. The system then orchestrates a sequential pipeline of four specialized Gemini-powered agents:

Extractor: Cleans and structures the raw data. Analyzer: Finds key patterns and anomalies. Insight: Cross-references the data to form high-level conclusions. Report Compiler: Drafts a polished final markdown report. Instead of passing the entire document to every agent, an underlying In-Memory RAG Layer chunks and embeds the data, serving only the top 5 most relevant context chunks to the agent based on its specific task. Finally, AgentDesk uses a custom Code-Execution MCP Server to dynamically write and execute matplotlib Python scripts, returning a natively rendered data visualization directly into the UI!

How we built it Because we had a strict ~3-hour time budget as a solo build, we had to be brutally efficient:

Frontend: Built natively using PyQt6 and Qt Quick/QML. This allowed us to quickly design a warm, dark-mode, animated dashboard without wrestling with heavy web frameworks or Electron. Agents: Written in Python using the google-genai SDK (gemini-3.1-flash-lite). They share a central "blackboard" state to pass information down the pipeline. RAG Architecture: Built a lightweight in-memory RAG layer using Gemini's text-embedding-004 and numpy cosine similarity, skipping the overhead of deploying a vector database. MCP Integration: We built a custom FastMCP server running over stdio that acts as a secure sandbox for the agents to run Python, intercepting plot images as Base64 strings and injecting them into the QML layout. Challenges we ran into Managing Token Limits: Initially, dumping full GitHub repositories into the context window for 4 consecutive agents was burning through tokens and slowing down response times. Implementing the on-the-fly RAG layer solved this, but required carefully intercepting the orchestrator flow so we didn't break the agents' existing logic. GUI Asynchrony: Keeping a smooth, 60fps Qt UI responsive while the Python backend was making heavy, blocking network requests to the Gemini API and running subprocesses for the MCP server. We solved this using Qt's QThread and asynchronous signals. Code Execution Sandbox: Capturing matplotlib outputs without GUI windows popping up on the host machine required forcing the Agg backend and capturing bytes directly in the MCP server process. Accomplishments that we're proud of The Speed of the Build: Going from a blank folder to a fully functioning multi-agent RAG application with a custom Code-Execution MCP server and a polished UI in under 3 hours! The Dashboard UX: The real-time visibility into the pipeline. Watching the status indicators pulse, the chat logs stream the RAG chunk selections, and the final visualization snap into place feels incredibly satisfying and premium. Preserving Native OS Feel: Adding native features like invoking the host OS's file dialogs (e.g., Dolphin on Arch Linux) for exporting the Markdown reports. What we learned The Model Context Protocol (MCP) is incredibly powerful when used locally over stdio. It allowed us to give our LLM a code-execution tool that felt completely decoupled and safe without needing complex Docker routing. You don't always need LangChain or heavy orchestration frameworks. A simple state dictionary (the "blackboard") and a linear pipeline in Python is often more predictable and easier to debug under a strict time constraint. What's next for AgentDesk Parallel Agents: Transitioning from a purely sequential pipeline to a directed acyclic graph (DAG) where the Extractor feeds multiple specialized Analyzers simultaneously. More MCP Tools: Plugging in more community MCP servers (like PostgreSQL or Slack integration) so the pipeline can fetch live data streams rather than just static documents. Export Formats: Adding support for exporting the compiled reports and visualizations directly into cleanly formatted PDF slide decks.

Built With

• python
• qml