Sketchmotion

• 

💡 Inspiration

Every AI storyboard tool in 2026 generates frames. Pretty pictures.

But nobody is solving the actual bottleneck — direction.

Creative teams sketch a board, then scatter the real work across Google Docs, Slack threads, and verbal feedback that evaporates between review cycles. By the time production starts:

• 🔴 Continuity is broken
• 🔴 Pacing intent is lost
• 🔴 The team is polishing the wrong beat

We built the tool that should have existed years ago — a storyboard workspace where direction, continuity, and revision stay attached to the board, not buried in a separate brief.

---

🎬 What it does

SketchMotion is a collaborative storyboard workspace with a GLM 5.1-powered Director Workflow.

🖼️ The Canvas

Arrange storyboard frames, set durations, add motion notes, collaborate in real time with live cursors and presence.

🎯 Direction Studio

Save creative intent alongside the board — mood, pacing, camera language, lighting, color, continuity constraints, and an avoid-list. This replaces the scattered direction notes that normally get lost between reviews.

🧠 GLM Director Workflow

One click sends the full board context to GLM 5.1. The model reasons over everything in a single structured pass and returns:

Output	What it does
📊 Storyboard Analysis	Frame-by-frame narrative breakdown
🎥 Shot Plan	Camera direction + timing recommendations
🔗 Continuity Rules	Catches drift before the team wastes time
🚀 Render Strategy	Production handoff guidance

♻️ Revision-Aware Planning

Type one concise note → hit Apply Revision → GLM 5.1 updates the entire plan without resetting from zero.

The board stays. The direction sharpens.

---

🛠️ How we built it

🖥️ Frontend React + TypeScript + Tailwind CSS + Vite — infinite canvas with pan/zoom, frame CRUD, sketch editing, drag-and-drop ordering, and real-time collaboration via Supabase Realtime.

⚡ Backend Supabase handles auth, Postgres persistence, file storage, and edge functions. Three edge functions power the AI pipeline:

• storyboard-plan — the Director Workflow
• generate-video — video generation path
• check-video-status — async status polling

🧠 GLM 5.1 Integration The Director Workflow runs server-side through the storyboard-plan edge function → calls GLM 5.1 via the Z.ai coding endpoint (/api/coding/paas/v4) with thinking enabled → returns a complete director's plan in one pass. Secrets never touch the client.

🏗️ Architecture Decision Feature-flagged the GLM path (VITE_AI_PROVIDER=zai) alongside the existing Google/Gemini workflow. The core product stays stable. The GLM path is metadata-first — no image fetches, no multimodal credits. Reliable and fast to demo.

🌐 Deployment Vercel (frontend) + Supabase hosted (edge functions + database)

---

🎯 Why GLM 5.1 specifically

This project demands long-horizon structured reasoning. The model needs to hold:

• 🔄 Frame-to-frame relationships across the full sequence
• 🎨 Creative constraints from 6 different director controls
• ⏱️ Pacing logic that spans every beat
• 📝 Accumulated revision context from prior passes

…all in a single pass.

Prompt chaining would lose coherence. Multi-call orchestration would fragment the plan. GLM 5.1's coding endpoint with thinking enabled is what makes revision-aware planning possible without resetting context.

---

⚔️ Challenges we ran into

🔧 Provider abstraction — Supporting both Google/Gemini and Z.ai/GLM required a clean provider interface so the UI, edge functions, and state management don't care which model is active. Took several iterations to get right.

📐 Metadata-first pivot — Early versions tried fetching frame images for multimodal analysis. Slow, expensive, fragile. We pivoted to metadata-first (frame titles, order, durations, motion notes) — which turned out to produce better structured plans because it forces the model to reason about sequence and intent, not pixel content.

🔁 Revision coherence — The revision loop needed to feel like updating a document, not starting over. Solved by passing the full previous plan as context alongside the revision note, letting GLM 5.1 decide what to preserve and what to update.

⏰ Time pressure — Built in one week alongside coursework. Prioritizing one tight demo path — board → direction → plan → revision — kept scope sharp.

---

📚 What we learned

✅ GLM 5.1's long-context reasoning is genuinely strong for structured creative planning — you can't fake this with prompt chaining

✅ Metadata-first AI workflows are more reliable and faster than multimodal approaches for planning tasks

✅ Feature-flagging AI providers early saves enormous debugging time

✅ The best hackathon demos show one tight loop done well, not ten features done halfway

---

🔮 What's next for SketchMotion

Priority	Feature
🎬	Connect GLM shot plan output → automated video generation for a full storyboard-to-animatic pipeline
🖼️	Multi-frame render orchestration using the render strategy GLM already produces
👥	Team review workflows where reviewers annotate the plan directly
📄	Export the director's plan as a production-ready PDF handoff document

---

SketchMotion — Direct the storyboard before the render.

Built With

• gemini
• glm5.1
• next.js
• python