Inspiration 💡

Every visionary product—from Uber to Airbnb—started as a quick, messy wireframe scribbled on a cocktail napkin or a whiteboard.

Yet, turning that flash of inspiration into an interactive prototype usually takes weeks: recreating low-fidelity screens in Figma, scheduling alignment meetings, and waiting for developers to write boilerplate frontend code. For non-technical founders, students, and product managers, this friction often kills ideas before they can be validated.

We asked ourselves: What if physical paper could directly compile into running software?

We created GhostWriter—an autonomous multi-agent AI system that transforms rough, hand-drawn paper sketches into live, interactive mobile applications running on your smartphone in under 60 seconds.

What It Does 🚀

GhostWriter completely eliminates design software and setup friction:

Draw: Sketch your mobile screen layouts, buttons, cards, and input fields on plain paper using a pen or marker.
Snap & Upload: Take a photo with your phone and upload it through the GhostWriter web dashboard.
Autonomous Agent Pipeline: Four specialized AI agents parse the visual semantics, infer layout hierarchy, write idiomatic React Native components, and self-heal syntax errors in a live compiler environment.
Instant Live App: A browser simulator boots up, accompanied by an instant Expo QR code. Scan it to interact with your freshly coded app right on your physical smartphone.

How We Built It ⚙️

GhostWriter relies on an event-driven, multi-agent architecture built to run concurrently for maximum speed:

Frontend Dashboard: Built with Next.js, React, and TailwindCSS for image uploads, agent status streaming, and instant live code previews via Expo Snack / Sandpack.
Backend API: Orchestrated with high-performance FastAPI (Python).
Vision & Layout Parsing: Powered by Gemini 2.5 Flash / GPT-4o. Rather than simple OCR, we feed the model a structured component vocabulary to classify rough boxes into specific mobile UI primitives (Header, TextInput, ActionCard, BottomTabNav).
Autonomous Multi-Agent Workflow:
- Vision Agent: Detects handwritten shapes, scribbles, text labels, and intent.
- Layout Agent: Formats the layout into a clean hierarchical UI Abstract Syntax Tree (AST).
- Code Generation Agent: Converts the AST into clean, modular React Native / Expo code.
- Validation & Self-Healing Agent: Runs the generated code through an AST syntax and bundle check. If syntax errors or missing packages are caught, it feeds the compiler output back to the agent for an automatic self-repair loop without human intervention.

Challenges We Ran Into 🧗

Deciphering Messy Handwriting & Ambiguity: Humans sketch quickly—crossed-out boxes, crooked lines, and scribble text are the norm. We solved this with few-shot visual prompting that teaches the model to focus on functional design patterns and UI ergonomics over pixel-level symmetry.
Synthesizing Run-Ready Code: Large Language Models frequently hallucinate missing imports or deprecated styling props. Building an automated feedback loop between the compiler sandbox and the Code Agent was critical to achieve zero-crash prototypes.
Latency Optimization: Running sequential agent calls can take minutes. We tuned the pipeline to execute visual layout extraction and component dependency resolution concurrently, bringing total processing time down from over 2 minutes to well under 60 seconds.

Accomplishments That We're Proud Of 🏆

True Zero-Friction Workflow: Taking an idea from a blank sheet of physical paper to a functioning app running natively on iOS and Android devices without touching a code editor.
Self-Healing Code Loop: Successfully building an autonomous validation step that detects syntax errors and resolves them before presenting the app to the user.
Live Interactive Demo: Demonstrating that non-technical creators can test product mechanics without ever learning Figma or React.

What We Learned 🧠

Prompt Engineering for Computer Vision: Visual spatial grounding requires strict structural constraints and typed JSON schemas.
Agent Orchestration: Fine-grained, role-based agents (Vision vs. Layout vs. Code) produce significantly higher fidelity and fewer bugs than relying on a single, monolithic LLM prompt.
The Value of Rapid Prototyping: Validating user interaction on an actual touchscreen phone reveals UX flaws that are completely invisible when looking at static design mockups.

What's Next for GhostWriter 🔮

Multi-Screen Navigation: Link multiple paper sketches together with drawn arrows to generate full multi-screen user journeys with React Navigation.
Instant Backend Generation: Automatically wire form fields to a generated serverless database (Supabase / SQLite) for persistent data storage.
Voice & Style Tweaks: Enabling users to tap any screen and say "Make the theme dark mode and add a floating action button" for real-time iterative refinements.