Inspiration
Every kid has a paracosm — an imaginary world with its own rules. We wanted a way to say yours out loud and see it land on your real table, in your real room — and then, because no generator gets it perfect on the first try, hand you the tools to rebuild it like a real game designer.
What it does
Paracosm is a hands-first mixed reality game maker for Meta Quest 3.
- Generate: turn your palm up to summon the cosm — a tiny galaxy orb menu — and say your idea ("a moon harbor pinball game with a grumpy crab captain"). Your words become a dream bubble that bursts into a playable tabletop diorama on your real table.
- Build: every game opens paused in Build mode. Pinch anything glowing to move it, reshape a race track by dragging its control knobs, add props from a library of thousands of 3D models, copy, delete, undo. Placement is validated so the game stays playable ("Too close to the flippers", "Too tight a turn") and crowded drops are nudged to the nearest valid spot.
- Simulate: a bot test-plays your build so you can watch your design work.
- Play: play it with your own hands — no controllers required.
- Say it: change the game instantly without starting over: "make it night", "make it winter", "turn the ocean into chocolate", "drive a guitar instead", "nine lives and a row of five hearts", "best of five rounds", "three laps", "make it deeper", "make it taller" on the piece you're holding.
- Rule and item cards: every genre has a headline rule card (Laps to win, Extra ball, Lives) and an item card with equippable gear (Soft tires, Spring shoes, Storm springs) whose effects you can feel.
- 20+ original genres: pinball, kart racing, platformers (tabletop, side-scrolling and 3D), one-on-one fighting, party brawls, run-and-gun boss fights, a pet simulator, sports (soccer, basketball, hockey, football, baseball, golf), rhythm, quests, and room-scale games that use your actual room.
- Worlds with depth: underwater games become a glass water tank on your table, cities rise into skylines, mountains tower.
- Come back: save your worlds to a shelf of cartridges and pick up exactly where you left off.
Everything is designed seated, within arm's reach, hands-first.
How we built it
- IWSDK + WebXR + three.js (TypeScript, Vite), deployed as a web app on Vercel, running in the Quest browser with passthrough and hand tracking.
- A game is a recipe — structured data describing genre, rules, cast, setting and layout. Every genre is a deterministic fixed-step simulation plus a view that builds itself from the recipe, so voice edits, rule taps and hand edits all change the same data through one commit path with shared undo.
- Meta's Muse API turns prompts into recipes and transcribes voice; an offline remixer builds every game locally too, so Paracosm keeps working if the AI service is unavailable — the AI is the starting point, not the product.
- Build mode is a shared editing layer: each genre exposes "editables" (bumpers, track control points, props, spawns) with validation rules, ghosts, snapping and persistence.
- 3D pieces come from open libraries (Kenney, KayKit, Quaternius, and Polyfork models under CC BY 4.0), with code-built fallbacks for every piece and strict Quest triangle and draw-call budgets.
- Custom GLSL shaders for the galaxy cosm, underwater caustics and depth, and world effects.
Challenges we ran into
- Making generation faithful: "a basket ball game where the players are ducks" originally produced the wrong genre. We built a prompt-intent layer so the sport, the setting and the cast you name are always honored.
- Hands at miniature scale: pinching small pieces on a table is hard, so we added generous grab zones, snapping, nudging and clear reasons when a placement is rejected.
- Pinball physics at tabletop scale without the ball tunneling through flippers, even after players rearrange the table.
- Quest performance: merging and instancing geometry to cut draw calls roughly in half, and light versions of heavy models.
What we learned
Generation is only the first draft. The magic happens when people can reach in and change it with their hands — and see a bot play their design seconds later.
What's next
Tower defense and table tennis genres, a mech cockpit view, item and inventory editors, sharing your worlds with friends, and co-located multiplayer.
Credits
3D models by Kenney, KayKit and Quaternius (CC0), and Polyfork (CC BY 4.0, full credits at /polyfork/CREDITS.txt). All games, characters and designs are original.
Built With
- hand-tracking
- iwsdk
- meta-muse-api
- meta-quest-3
- mixed-reality
- node.js
- passthrough
- three.js
- typescript
- uikitml
- vercel
- vite
- webxr