GameVibe

Learning tool to help children build a passion for vibe coding in a way they love—video games.

Comment

GameVibes

Inspiration

Every kid has a game idea — a platformer with dragons, a shooter on Mars, a racing game through their neighborhood. But the gap between imagining a 3D game and building one is massive. Traditional game engines like Unity or Unreal have steep learning curves: scene graphs, shaders, asset pipelines, C# or C++ boilerplate. Even "beginner-friendly" tools like Scratch cap out at 2D.

We asked: what if building a 3D game was as easy as describing it? What if you could say "build a platformer with a blue sky and green ground" and watch it appear — then say "add a dragon" and get an actual 3D model dropped into your scene?

The rise of foundation models made this feel possible for the first time. Large language models can write Three.js code. Image-to-3D pipelines (like Meshy.ai) can turn a text prompt into a .glb model. Audio generation (ElevenLabs) can produce sound effects from a description. We wanted to unify all of these into a single, cohesive IDE — a Replit meets Scratch, but for 3D games, powered entirely by AI.

What It Does

GameVibes is a browser-based 3D game development IDE where you build Three.js games by chatting with an AI assistant. The core loop is:

  1. Describe what you want — the AI writes Three.js code
  2. Watch the game render live in an embedded preview
  3. Iterate — ask for changes, new assets, sound effects, gameplay tweaks

Under the hood, the platform orchestrates several AI systems:

AI Coding Agent

Claude Sonnet 4.6 (with extended thinking and tool-calling) acts as the game developer. It has access to 10 tools: read_file, create_file, edit_file, list_files, bash, generate_sprite, generate_sound, list_sprites, get_console_logs, and refresh_game. Each tool call is streamed to the user in real-time via SSE with color-coded diffs and terminal-style output blocks.

3D Sprite Generation

A two-stage pipeline:

  • Stage 1 (Preview): Flux generates a 2D concept image in about 5 seconds. The user sees the preview and clicks confirm or reject.
  • Stage 2 (3D Model): Upon confirmation, Meshy.ai (meshy-6) generates an untextured 3D .glb model in about 61 seconds. An optional PBR texture refine pass takes an additional 80 seconds.

The full pipeline from prompt to usable 3D model averages roughly 79 seconds plus user confirmation time (5s for Flux + 61s for Meshy + 3s for download + 10s for render).

After generation, Puppeteer renders 6-angle preview screenshots for the asset panel.

AI Sound Effects

ElevenLabs generates sound effects (gunshots, footsteps, ambient audio) from text descriptions in about 1 second, saved as .mp3 files immediately usable in game code via new Audio("/models/name.mp3").play().

Voice Brainstorm

An ElevenLabs conversational agent lets you brainstorm game ideas via voice. When you say "build it," the agent calls a build tool that hands your idea off to the coding AI.

Sandboxed Execution

All AI-generated code runs inside a Docker container (python:3.12-slim) with hard resource limits: 512 MB memory, 0.5 CPU cores, 64 max PIDs, and no network access.

Accounts and Projects

Email/password auth with JWT + bcrypt, multiple projects per user, full chat history persisted to SQLite.

How We Built It

The architecture follows a three-panel IDE layout (game preview | assets | chat) served by an Express.js/TypeScript backend on port 8080.

Frontend: Vanilla JS and CSS with an OpenCode-inspired dark theme. No React, no build step — just static HTML files (ide.html, ide.js, ide.css) served directly. The game preview lives in an iframe pointed at the Docker sandbox's port 3000.

Backend (server.ts, ~934 lines): A single Express server handles:

  • JWT authentication (signup/login/logout)
  • Project CRUD and per-project workspace management
  • SSE-streamed chat endpoint (/api/p/:id/chat) that orchestrates the full AI tool-calling loop
  • Proxy endpoints for Meshy.ai 3D generation, Flux 2D previews, and ElevenLabs sound generation
  • Puppeteer-based 6-angle GLB rendering for sprite thumbnails

AI Integration: We route through the NanoGPT API to access Claude Sonnet 4.6 with thinking enabled. The system prompt (SKILL.md, ~376 lines) includes camera recipes, movement code templates, collision detection patterns, and strict rules to prevent common Three.js bugs (inverted mouse axes, missing pointer lock, objects below y=0, etc.). The AI sees all 10 tool definitions and uses tool_choice: auto.

Sandbox (sandbox.ts): Each project gets a dedicated workspace directory bind-mounted into the Docker container. The sandbox manager handles container lifecycle, project switching with debouncing (minimum 2 second delay), path sanitization to prevent directory traversal, and command blocking for destructive operations.

Database: SQLite via better-sqlite3 stores users, projects, chat messages, and sprite metadata (name, description, Meshy task ID, GLB URL, status, dimensions).

Key dependencies: Express 5, Three.js r183, Puppeteer, Playwright, @11labs/client SDK, better-sqlite3, bcryptjs, jsonwebtoken.

Challenges We Ran Into

Docker sandbox thrashing. Early on, every chat message triggered a container restart — if the user sent messages quickly, Docker would be spinning up and tearing down containers hundreds of times. We fixed this with project-aware switching (only restart when the project actually changes) and a 2-second debounce on switchProject.

Sprite path hallucination. The AI kept guessing sprite filenames like /models/dragon.glb instead of the actual Meshy-generated paths like /models/sprite_018faa.glb. We had to add explicit rules in the system prompt ("ALWAYS call list_sprites first") and built the list_sprites tool to return exact paths with bounding box dimensions (width x height x depth).

Meshy timing and stuck progress. Meshy-5 would stall at 88% progress for about 80 seconds before finishing. We benchmarked both versions and switched to meshy-6, which was 2.3x faster for preview generation (61 seconds vs 143 seconds). We also built a polling system that survives DOM rebuilds using JS objects instead of DOM state.

Console error forwarding. When the AI's generated code has a runtime error, the error happens inside an iframe inside a Docker container — three layers deep. We inject a reporter script into game.html on every write that patches console.log, console.warn, and console.error along with window.onerror to postMessage errors back up to the parent frame, where the IDE captures them for the get_console_logs tool.

Security. We did a full security audit and patched: path traversal in the sandbox (now sanitized with posix.normalize + prefix stripping), stale JWT tokens causing foreign key constraint errors after DB resets, and command injection via the bash tool (blocked patterns like rm -rf /, mkfs, dd if=, and fork bombs).

Accomplishments That We're Proud Of

  • End-to-end AI game creation in under 2 minutes. From a text prompt to a playable 3D game with custom 3D models and sound effects — no code written by the user.

  • The sprite generation UX. The two-stage confirm flow (fast 2D preview followed by slow 3D generation) respects the user's time. You see what you're getting in about 5 seconds before committing to the 60-second 3D pipeline. The 6-angle Puppeteer renders give a real sense of the model before it's in-game.

  • Real sandboxing. The Docker container has no network access, capped memory/CPU/PIDs, and all file operations go through a sanitized API. The AI literally cannot escape the sandbox.

  • A 376-line system prompt that actually works. The SKILL.md includes camera recipes, movement code, collision detection templates, and a "DO NOT" list of the top 10 bugs we encountered. This turned the AI from a mediocre code generator into a reliable game builder.

  • 33 commits, zero frameworks. The entire frontend is vanilla JS/CSS. No React, no webpack, no build step. It just works.

What We Learned

  • System prompts are the product. The difference between a broken game and a working one came down to the quality of SKILL.md. Iterating on the AI's instructions was as important as writing backend code. Specific, opinionated rules ("DO NOT invert mouse axes", "DO NOT place objects below Y=0") outperformed vague guidance every time.

  • Two-stage generation is the right UX for slow pipelines. When 3D model generation takes 60+ seconds, you need a fast preview gate. The Flux to confirm to Meshy flow cut wasted generation time dramatically.

  • Benchmarking APIs matters. Switching from meshy-5 to meshy-6 saved 82 seconds per sprite (143 seconds down to 61 seconds) with no quality loss. We only discovered this by running a controlled benchmark.

  • Docker sandbox isolation is non-negotiable for AI code execution. Letting an LLM write and execute arbitrary code requires real isolation — not just input validation. Memory limits, PID limits, network disabling, and command blocking are all necessary layers.

  • SSE streaming with tool-call visibility builds trust. Users can see every edit_file diff, every bash output, every generate_sprite call as it happens. This transparency makes the AI feel like a collaborator rather than a black box.

What's Next for GameVibes

  • Multiplayer. WebSocket-based multiplayer so users can build and play games together in real-time. The Docker sandbox already runs a web server — extending it to sync game state between players is the natural next step.

  • Asset library and sharing. A persistent gallery of community-generated 3D sprites and sound effects so users don't have to regenerate common assets (trees, coins, explosions) from scratch every time.

  • Export to standalone. One-click export of the finished game as a self-contained HTML file (or deployable to Vercel/Netlify) so users can share their games with anyone.

  • Mobile support. Touch controls and a responsive IDE layout so users can build and play on tablets and phones.

  • Fine-tuned game AI. Training a specialized model on Three.js game code to reduce hallucinations, improve first-try success rate, and handle more complex game mechanics (physics, pathfinding, particle systems) without extensive prompt engineering.

  • Collaborative editing. Real-time collaborative chat where multiple users can talk to the same AI and build a game together, with conflict resolution on file edits.

Built With

  • bash-frameworks-&-libraries:-express.js-5
  • bcrypt-(bcryptjs)
  • claude
  • containers
  • cookie-parser-other:-sse-(server-sent-events)-for-real-time-streaming
  • css
  • docker
  • docker-(colima/docker-desktop)-databases:-sqlite-(via-better-sqlite3)-apis:-nanogpt-api-(routing-to-claude-sonnet-4.6-with-thinking)
  • elevenlabs-(sound-generation-+-conversational-ai)
  • flux-(via-nanogpt
  • for-2d-image-previews)-auth/security:-jwt-(jsonwebtoken)
  • html
  • javascript-(vanilla)
  • languages:-typescript
  • memory/cpu/pid
  • meshy.ai-(meshy-6-for-text-to-3d)
  • playwright
  • puppeteer
  • sandboxing
  • socket.io
  • three.js-r183
  • with
Share this project:

Updates