Gemini-Sketch-To-Game

• 

  Home Page

• 

  example level 1

• 

  example level 2

Inspiration

Wanted a 5-second mood refresh between coding sessions. The perfect loop: Doodle -> AI magic -> Play -> Repeat.

Inspired by napkin sketches turning into real games, but wanted zero friction - no editors, just pure Gemini 3 vibes.

What it does

Upload any sketch -> Instant playable platformer.

Drag napkin doodle (player/platforms/enemies/goal)

Gemini 3 analyzes -> Generates physics-ready LevelConfig

Phaser 3 renders: smooth jumps, collision, win/lose

←→ move, ↑ jump (ground only), Restart Button

How we built it

Frontend (React/Vite): SketchUploader handles drag-drop + API key (stored in localStorage). Minimal UI with analyzing states.

geminiService.ts: Converts sketch -> base64 -> sends to gemini-3-flash-preview with strict JSON schema prompt. Forces "GUARANTEED PLAYABLE" levels by requiring logical platform progression.

GameEngine.tsx (Phaser 3): Takes LevelConfig -> renders physics world:

• Ground/platforms from obstacles[]

• Static red triangles from hazards[]

• Green goal block

• Blue player with momentum physics (velocity.x * 0.85)

• Procedural parallax clouds (no assets needed)

Challenges we ran into

Physics hell: Player "stuck" to ground -> setFriction(0) + setDrag(500) Continuous jumping -> player.body!.touching.down only Hazards falling -> physics.add.staticSprite() + refreshBody() Impossible levels -> Prompt engineering

Accomplishments that we're proud of

Minimalism - 3 clicks from sketch to victory Procedural clouds - Parallax skybox, zero assets Perfect controls - Momentum feels pro, no double jump spam Hackathon demo - "Draw this -> Instant game" wows judges

What we learned

1. Gemini 3 multimodal - scary good at sketch understanding
2. Phaser 3 - perfect for rapid prototypes
   
3. Strict JSON prompts - reliable AI output
4. Hackathon truth: Ship minimal + polish controls > feature creep

Result: Clean 5-minute brain break. Draw, jump, win, repeat. ☁️✨

What's next for Gemini-Sketch-To-Game

• Mobile touch controls
• Score/time system ("Beat your best sketch")
• Level sharing ("Play my doodle!")
• Patrolling enemies (red triangles chase you)
• Chrome extension - Screenshot -> Instant game tab

Built With

• css
• gemini
• javascript-frontend:-vite
• phaser.js
• react
• typescript
• vite