SimVerse

SimVerse is a sketch-first physics sandbox where you “draw on reality” and describe intent in natural language. The inspiration came from noticing how many hackathon AI projects collapse into yet another chatbot UI, I wanted something that feels hands-on and visual, draw a region, draw an arrow, then ask for “a swirling motion here” and watch the simulation change immediately. I learned a lot about turning language into safe, executable control, especially how important it is to force an LLM to output strict structured actions instead of free-form text.

I built SimVerse as a web app with a 2D physics world running in real time. Users sketch regions and arrows directly on the canvas, those sketches become constraints (where forces apply) and hints (direction/intent). When you type a command, Gemini 3 produces a JSON “action plan” (e.g., add a vortex, apply wind in a region, tune damping/restitution). The app validates the JSON against a schema, clamps values for stability, and applies actions through a controller layer so the UI never touches the physics engine directly.

The hardest challenges were reliable sketch interaction (pointer events + previews + deletion), keeping the simulation stable under extreme forces, and making Gemini consistently return valid JSON with clear explanations.

Built With

• css
• gemini3
• google-ai-studio
• html5
• matter.js
• react
• typescript
• vite