PaintTheStory.NFT

PaintTheStory.NFT

Comment

Inspiration

Creativity is often a solo experience — drawing, writing, and storytelling usually happen in isolation. We wanted to turn that into something collaborative, interactive, and expressive.

At the same time, we were inspired by how AI can interpret human creativity. What if a simple drawing could become a story? What if that story could be spoken? What if all of it could live on-chain?

More importantly, we were thinking about children today. Many kids spend hours on passive gaming or content consumption. We wanted to create something different — a space where they can:

🎨 actively create instead of just consume

🤝 collaborate with others in real time

📖 turn imagination into stories

🎙️ hear their creativity come to life

PaintTheStory encourages kids to draw, imagine, and build stories together, transforming screen time into something creative, social, and meaningful.

It blends:

human creativity (drawing), AI imagination (story + transformation), and ownership (NFTs),

into one playful and collaborative experience.

What it does

PaintTheStory is a real-time collaborative drawing platform where users can:

🎨 Draw together in live shared rooms

✨ Use AI to transform their drawings into enhanced images

📖 Generate stories from their artwork

🎙️ Listen to their story using AI voice narration

🪙 Mint their creations as Solana NFTs

Users can also:

create and join public rooms see active rooms collaborate in real-time turn art → story → voice → NFT in one flow How we built it

Frontend

Next.js + TypeScript Tailwind CSS + shadcn/ui Custom collaborative canvas with real-time drawing sync

Backend

Next.js API routes MongoDB Atlas for room storage

Real-time Collaboration

WebSocket-based room system for: strokes cursors canvas sync

AI Pipeline

Gemini: image understanding (canvas → structured description) image generation / transformation DigitalOcean Gradient AI: story generation from structured scene + user prompt ElevenLabs: text-to-speech narration of generated stories

Blockchain

Solana (Devnet) Metaplex UMI SDK for NFT minting IPFS (Pinata) for image + metadata storage

MLH Track - Solana

We integrated Solana to enable:

minting collaborative artwork as NFTs assigning ownership to users’ wallets storing metadata (image + story)

Each piece of art becomes a permanent, verifiable digital asset.

MLH Track - Gemini

We used Gemini for:

analyzing drawings from the canvas extracting structured scene descriptions generating or transforming images

Gemini acts as the “visual intelligence layer” of our app.

MLH Track - MongoDB

MongoDB Atlas is used to:

store active rooms track room creators (wallet addresses) enable listing of public rooms

This allows a scalable and dynamic “Open Rooms” discovery system.

MLH Track - ElevenLabs

We used ElevenLabs to:

convert AI-generated stories into speech provide immersive storytelling experiences

Users can hear their drawings come to life, not just read them.

Challenges we ran into

Handling real-time canvas sync without lag or conflicts Making AI-generated images properly apply to the canvas Managing multiple AI pipelines (Gemini → Gradient → ElevenLabs) Dealing with API limitations (voice restrictions, key revocations) Preventing state overwrites between canvas strokes and AI updates Ensuring wallet integration worked smoothly across components Accomplishments that we're proud of Built a full end-to-end creative pipeline: drawing → AI → story → voice → NFT Successfully integrated multiple AI systems together Enabled real-time multi-user collaboration Created a smooth UX combining Web3 + AI + creativity Designed a project that is both fun and technically complex

What we learned

How to orchestrate multiple AI services in one pipeline Handling real-time UI + state synchronization Building full-stack apps with Next.js API routes Working with Solana NFT tooling (Metaplex UMI) Managing API constraints and fallbacks effectively

What's next for PaintTheStory

🎭 Add story styles (fantasy, horror, sci-fi, etc.) 🧑‍🤝‍🧑 Add user accounts + saved galleries 🖼️ Layered canvas (AI background + user drawings) 🔊 Voice selection and customization 🌍 Public gallery of minted artworks 📱 Mobile optimization and touch-first UX

Built With

Share this project:

Updates