Haunted Homes AI Agents

• 

  Haunted House AI Agent

Inspiration

My family and I are currently searching for an apartment to live in. After spending countless hours browsing multiple listing sites, comparing neighborhoods, and researching nearby amenities, I realized this tedious process was perfect for automation. I decided to build an AI agent that could do all this work for me—searching listings, analyzing locations, and presenting everything in one comprehensive report.

What I Learned

• Multi-Agent Architecture: How to coordinate multiple specialized AI agents using the Deep Agents framework (LangGraph), where each agent handles a specific task (property search, location analysis, image decoration).
• Human-in-the-Loop Workflows: Implementing interrupt mechanisms that pause agent execution for user approval before proceeding with expensive operations.
• Context Management: Using filesystem backends to offload large data (property details, images) and prevent context window overflow.
• API Integration: Connecting Tavily for web search, Google Places for location data, and Gemini Vision for image analysis and generation.

How I Built It

Backend (Python/FastAPI)

• Created a supervisor agent that orchestrates three sub-agents: Property Search, Location Analysis, and Halloween Decorator.
• Used Deep Agents' create_deep_agent with FilesystemBackend for shared storage between agents.
• Implemented human-in-the-loop using interrupt_on configuration for property review.
• Built REST endpoints (/api/invoke, /api/resume, /api/state) protected by Clerk authentication.

Frontend (Next.js/React)

• Built a 3-step workflow: Search Form → Property Review → Final Report.
• Integrated Clerk for authentication with protected routes.
• Created components for property selection and comprehensive report display with location analysis and AI-decorated images.

Challenges Faced

• Context Overflow: The agents initially tried to pass large base64 images through the conversation, causing context limits to be exceeded. Solved by saving decorated images to external disk and only passing metadata references.
• HITL Interrupt Format: Understanding the Deep Agents interrupt payload structure took debugging—the action_requests format differs from simple interrupt values.
• State Synchronization: Ensuring the frontend correctly tracks thread IDs and resumes the right conversation after user approval required careful state management.
• Agent Coordination: Getting sub-agents to write data to shared filesystem locations that the supervisor could later read required proper FilesystemBackend configuration with absolute paths.

Built With

• clerk
• deep-agents
• docker
• fastapi
• gemini-vision-api
• google-places
• langgraph
• next.js
• pydantic
• python
• tailwind-css
• tavily-api
• typescript