GhostTrace AI
The Inspiration
Most automation tools require people to manually identify a process, record every step, and build a workflow. But in the real world, people rarely work from perfectly documented processes, they simply get the job done.
We wanted to reverse this approach: What if software could watch how people work, understand what they repeatedly do, and build the automation itself?
That question became GhostTrace AI.
What We Built
GhostTrace AI is an autonomous process-intelligence and digital-workforce platform. It observes workflows across applications, identifies repetitive patterns, reconstructs them as visual workflows, and converts them into semantic “Workflow DNA.”
From there, AI generates executable automation using Playwright, validates it in a sandbox, and monitors execution. When something changes or breaks, the system can reason about the failure, attempt to self-heal the workflow, and request human approval when necessary.
Once approved, the workflow can become a Digital Employee capable of repeatedly executing the task.
The core idea is simple:
«GhostTrace watches once, learns the workflow, and works for you.»
How We Built It
We designed GhostTrace as a multi-agent architecture where specialized agents collaborate instead of relying on a single AI component.
- Observation Agent — captures and interprets user actions.
- Workflow Discovery Agent — identifies repetitive patterns.
- Workflow Reconstruction Agent — converts actions into structured workflows.
- Workflow DNA Engine — represents the underlying business logic and intent.
- Automation Agent — generates Playwright-based executable workflows.
- Validation Agent — tests automations in a controlled environment.
- Self-Healing Agent — diagnoses runtime failures and adapts workflows.
- Human Approval Layer — keeps humans in control before deployment.
- Automatic Demo — executes approved workflows continuously.
The backend uses FastAPI, Python, SQLite, Playwright, and LangGraph, with Gemini providing the reasoning capabilities. The frontend uses React/Next.js, React Flow, Framer Motion, and Monaco Editor to visualize workflows and automation logic.
What We Learned
Building GhostTrace taught us that creating an AI agent is much easier than building a reliable agentic system.
We learned how important orchestration, state management, validation, error handling, and human-in-the-loop decisions are. We also learned that giving an AI more autonomy isn't always better, the system needs clear boundaries and checkpoints to remain trustworthy.
Most importantly, we learned to think beyond individual AI responses and design systems where multiple agents can observe, reason, act, validate, and recover together.
Challenges
The biggest challenge was making the system reliable when real-world workflows are unpredictable. Small changes in UI elements, unexpected application states, or failed actions could break an otherwise correct automation.
We addressed this by introducing validation, structured workflow representations, runtime monitoring, and a self-healing layer rather than assuming that generated automation would always work perfectly.
Another challenge was coordinating multiple agents without creating unnecessary complexity. Designing clear responsibilities and communication between agents became essential to keeping the architecture understandable and scalable.
What's Next
GhostTrace is designed around a larger vision: software that doesn't just automate workflow, it discovers them.
Instead of asking users to learn automation tools, GhostTrace aims to let organizations turn the way their people already work into intelligent, adaptable digital processes.
The future of automation shouldn't require humans to teach machines every step. Sometimes, the machine should simply watch, understand, and learn.
Log in or sign up for Devpost to join the conversation.