Deptheon AI

Demo:(there are two video because I do not have a loom premium so my old loom hit a 5 min limit) https://www.loom.com/share/6c3a36b5b307436292d1b4e9302cafd9 (this is the final video tho((Main) https://www.loom.com/share/d699ee5d3c43420d8391df55fd936a6e

The World's Most Capable Autonomous AI Agent

Deptheon can autonomously orchestrate complex, multi-hour workflows involving web research, phone calls, emails, code execution, and access to 3000+ tools - all while you sleep.

🎯 What is Deptheon?

Deptheon is a fully autonomous AI agent that won 1st place at the Agents in the Loop Hackathon 2025 in San Francisco. Unlike traditional chatbots that only handle web-based tasks, Deptheon can:

• 🌐 Conduct comprehensive web research using advanced search and scraping
• 📞 Make live phone calls and conduct interviews using AI voice technology
• 📧 Send emails and manage communications across multiple platforms
• 💻 Execute code and automate tasks in sandboxed environments
• 🔗 Access 3000+ tools and APIs through Composio integration
• 🔄 Chain complex workflows autonomously for hours without human intervention

🚀 The Demo

During the hackathon, Deptheon showcased its capabilities with an autonomous multi-modal workflow that would typically require hours of manual work:

What Deptheon Accomplished (Autonomously)

1. 🔍 Web Research

   • Searched for information about the "Agents in the Loop" hackathon
   • Scraped relevant websites and documentation
   • Found the DevPost page and event details

2. 📞 Live Phone Interview

   • Automatically dialed +1-415-605-6693
   • Conducted a professional interview using VAPI's AI voice technology
   • Asked structured questions about the hackathon experience
   • Recorded and transcribed the entire conversation

3. 📝 Content Creation & Email

   • Analyzed the interview transcript
   • Composed a professional LinkedIn post highlighting the achievement
   • Sent the draft via Gmail to the winner for review and posting

Key Interview Insights Captured

Project: "Deptheon" - A fully autonomous agent with access to hundreds of tools

Inspiration: Long-time fascination with agentic AI and perfect hackathon theme fit

Technical Challenge: Integrating diverse toolsets smoothly across different platforms

Innovation: Full autonomy across planning/execution cycles with real-world actions

Future Vision: Scale to more complex workflows and open beta access

Advice: "Just go and have fun! The magic happens when curiosity leads the way."

🏗️ Architecture Overview

Deptheon is built on a sophisticated tool-based agent architecture that enables seamless integration of multiple capabilities:

┌─────────────────────────────────────────────────────────────┐
│                       DEPTHEON CORE                        │
├─────────────────────────────────────────────────────────────┤
│  🧠 LLM Engine (OpenAI o3-2025-04-16)                     │
│  🎯 Autonomous Planning & Execution                        │
│  🔄 Multi-step Workflow Orchestration                      │
└─────────────────────────────────────────────────────────────┘
                              │
            ┌─────────────────┼─────────────────┐
            │                 │                 │
    ┌───────▼──────┐ ┌────────▼────────┐ ┌─────▼─────┐
    │   COMPOSIO   │ │      VAPI       │ │  PYTHON   │
    │   3000+      │ │   Voice AI      │ │ EXECUTION │
    │   Tools      │ │   Calling       │ │ SANDBOX   │
    └──────────────┘ └─────────────────┘ └───────────┘
            │                 │                 │
    ┌───────▼──────┐ ┌────────▼────────┐ ┌─────▼─────┐
    │   • Gmail    │ │  • Outbound     │ │ • Web     │
    │   • GitHub   │ │    Calls        │ │   Scraping│
    │   • Slack    │ │  • Interview    │ │ • Data    │
    │   • Notion   │ │    Conduct      │ │   Analysis│
    │   • And      │ │  • Transcript   │ │ • API     │
    │     2996+    │ │    Capture      │ │   Calls   │
    └──────────────┘ └─────────────────┘ └───────────┘

💡 Key Features

🤖 Fully Autonomous Operation

• No human intervention required during task execution
• Intelligent planning and adaptive problem-solving
• Self-directed workflow orchestration

🛠️ Extensive Tool Integration

• Composio: Access to 3000+ tools and APIs (Gmail, GitHub, Slack, Notion, etc.)
• VAPI: AI-powered voice calling and conversation capabilities
• Python Execution: Sandboxed code execution for data processing and automation
• Web Research: Advanced search and content extraction

🔄 Agents-in-the-Loop Architecture

• Continuous planning → execution → learning cycles
• Dynamic tool selection based on task requirements
• Real-time adaptation to changing conditions

How I Built It Architecture Design I started with a tool-based agent architecture where everything the agent can do is encapsulated in discrete, composable tools: class Deptheon(ToolCallAgent): available_tools: ToolCollection = Field( default_factory=lambda: ToolCollection( PythonExecute(), # Code execution & web scraping ComposioTool(), # 3000+ API integrations
VapiTool(), # AI voice calling DateTimeTool(), # Time management Terminate(), # Graceful completion ) ) Key Technical Decisions

1. Sandboxed Execution Environment [sandbox] use_sandbox = true image = "python:3.12-slim" memory_limit = "1g" cpu_limit = 2.0 timeout = 300 Safety was paramount - the agent needed to execute arbitrary code without compromising the host system.
2. Advanced Prompting Strategy Instead of simple instruction-following, I designed prompts that encourage autonomous decision-making: "You cannot ask the user anything, you have to do everything by yourself and if you feel you have accomplished the task, you can end the conversation."
3. Tool Abstraction Layer Every tool implements the same interface, making them composable: async def execute(self, *, name: str, tool_input: Dict[str, Any] = None) -> ToolResult: 🚧 Challenges I Faced Challenge 1: Authentication Hell Problem: Managing authentication for 3000+ tools is a nightmare. Different OAuth flows, API keys, connection IDs... Solution: Built a persistent connection management system that stores authentication tokens and automatically handles refresh cycles: _CONNECTION_STORE_PATH = Path(os.path.expanduser("~/.composio_connections.json")) Challenge 2: Real-Time Voice Integration Problem: Making the agent capable of actual phone conversations, not just text-based interactions. Solution: Integrated VAPI's AI voice platform with dynamic assistant configuration: assistant_config = { "firstMessage": "Hi, congratulations on winning the hackathon!", "systemPrompt": "You are conducting an interview...", "voice": "andrew", "model": "gpt-4o-mini" } Challenge 3: Workflow State Management Problem: How do you maintain context across web research → phone calls → email composition when each uses completely different APIs? Solution: Implemented a stateful agent base class that maintains conversation history and intermediate results across tool executions. Challenge 4: Autonomous Decision Making Problem: Most agents need explicit instructions for each step. How do you make them truly autonomous? Solution: Used OpenAI's o3 reasoning model with carefully crafted system prompts that encourage proactive problem-solving rather than reactive responses.

Built With

• agent
• composio-(3000+-apis)
• openai-o3-reasoning-model
• python-3.12+
• vapi
• voice