ScribbleScript

Inspiration

I’ve had many moments where I sketch a cool UI idea in a random notebook page, only to have that momentum disappear the second I open a blank code editor. Moving from a rough drawing to an actual prototype usually takes forever. I wanted to build something that matches the speed of human creativity, turning a messy scribble into a live, working React app in seconds.

What it does

ScribbleScript is the bridge between my pen and my production code. You just snap a photo of a hand drawn sketch and upload it. The engine, powered by Gemini 3 Flash, reads the handwriting and layout to generate clean React code using Tailwind CSS. It doesn't just show a picture; it renders a functional, interactive prototype right in the browser.

How we built it

Frontend was built with Next.js and Tailwind CSS. I hooked up Google Gemini 3 Flash via the Vercel AI SDK to handle the multimodal heavy lifting. I used Sandpack to create a live coding environment so the AI’s code actually runs and renders instantly. And I pulled in Lucide-React to make sure the generated UIs looked polished.

Challenges we ran into

The core difficulty was semantic interpretation. Gemini 3 is powerful, but translating a messy pencil circle into a high-end avatar rather than a generic button required a strict Design Token framework. I moved from basic prompting to enforcing a "Senior UI Engineer" persona that prioritizes layout hierarchy and accessibility. I also had to optimize multimodal payloads to bypass rate limits while maintaining enough resolution for the AI to read my handwriting.

Accomplishments that we're proud of

Successfully bridging the gap between raw pixels and executable JSX. Engineering the system to consistently output a modern, "Vercel-style" dark mode, complete with glassmorphism and motion from analog scribbles. And moving past static mockups. Seeing a sketched toggle actually maintain React state and respond to clicks felt like true magic.

What we learned

I mastered the architecture of multimodal data mapping, specifically the process of translating visual spatial cues from an image into a structured React DOM hierarchy. This involved bridging the gap between raw vision data and executable JSX, ensuring the output adhered to professional layout standards. Additionally, I learned to manage real-time sandboxing by building a secure pipeline to sanitize and inject dynamic code strings into a virtualized execution environment, a critical step in maintaining performance while synchronizing a remote model with a live browser preview.

What's next for ScribbleScript

Multi-Page Flows by uploading a whole stack of sketches to build a full app flow. Teaching the engine to use specific libraries like Shadcn/UI. Letting users "talk" to their sketch to add features on the fly.

Built With

• gemini
• lucid-react
• next.js
• sandpack
• tailwind
• typescript
• vercel