🎨 DoodlePop

Empowering kids to bring their imagination to life — from doodle to 3D world.

🌟 Inspiration

At DubHacks, our team was inspired by the XR booth’s Vision Pro demos — a glimpse into the future of immersive creativity. We wanted to harness that power for something meaningful: helping kids express their imagination. DoodlePop lets children sketch their ideas and instantly see them transformed into interactive 3D objects, fostering creativity through technology.

🚀 What It Does

DoodlePop transforms 2D sketches into interactive 3D models in real time. Users draw on an iPad or upload a sketch, and the app automatically generates a 3D object viewable in Apple Vision Pro, letting kids explore their creations in spatial reality.

🧠 How We Built It

DoodlePop integrates AI, iOS development, and spatial computing into a seamless creative pipeline.

🖊️ iPad Frontend

Built with SwiftUI for a clean, modern interface
PencilKit for natural Apple Pencil sketching
SwiftData + Combine for reactive local persistence and state management
Automatically uploads sketches to Firebase after each session

⚙️ Backend & Processing

FastAPI (Python) — REST endpoints handle image uploads and AI model calls
Firebase — Firestore stores metadata, Firebase Storage hosts images and 3D assets
Gemini Nanobanana API — enhances the sketch into a more detailed image
Tripo3D API — generates 3D geometry from the enhanced image
GLB → USDZ converter — prepares the model for Apple Vision Pro compatibility

🕶️ VisionOS App

Swift + RealityKit for immersive 3D rendering
Fetches USDZ files from Firebase Storage
Supports interactive physics, drag gestures, and multi-object exploration

🧩 Tech Stack Summary

Layer	Technologies
Frontend (iPad)	SwiftUI, PencilKit, SwiftData, Combine
Spatial App	RealityKit, visionOS SDK
Backend	FastAPI (Python), Firebase (Firestore + Storage), Gemini API, Tripo3D API
3D Formats	GLB → USDZ conversion
Tools	Xcode, Python, GitHub, Firebase CLI

⚔️ Challenges

Integrating multiple 3D APIs (Tripo3D + Gemini) with incomplete documentation
Troubleshooting Firebase Cloud Functions in Python (new feature!)
Managing real-time file conversions (GLB → USDZ) across platforms
Balancing time between iPad UX, VisionOS rendering, and backend logic in just 24 hours

🏆 Accomplishments

End-to-end working prototype: sketch → image → 3D model → Vision Pro viewer
Cross-platform pipeline linking SwiftUI, FastAPI, and Firebase
Learned how to implement real-time AI-driven creativity tools
Built everything from scratch within 24 hours — and stayed up until sunrise doing it 🌅

📚 What We Learned

Full-stack XR development — from iPad sketching to Vision Pro rendering
How to design for user creativity instead of just functionality
The importance of early planning (we spent 3+ hours ideating — next time, we’ll move faster!)
How AI and XR can merge to create meaningful, educational tools for children

🌈 The Mission Ahead

DoodlePop is more than a hackathon project — it’s a step toward inspiring creativity in the next generation. We envision classrooms and kids’ spaces where anyone can draw an idea and instantly explore it in 3D, blending art, imagination, and technology.