Physics Lab

Physics Lab turns a physics scenario into a live experiment. Type it in plain English, and AI builds a simulation with physics engine you can run, slow down, pause and inspect.

Comment

Inspiration

Ever since school I've loved physics, and I solved problems by imagining them first: the block sliding down the ramp, the ball leaving the table. I always wanted an app that could put that picture on the screen and make physics accessible and fun for everyone. With LLMs that can write structured scenes and a real physics engine that runs in the browser, I could finally build it.

What it does

Physics Lab turns a description of a mechanics situation into a 3D experiment you can run.

  1. You describe the situation in plain English, with any numbers you know.
  2. An AI agent builds the scene: bodies, joints, springs, cords, friction, contacts and timed actions.
  3. Before you see it, the lab validates the scene, runs the first half-second of motion in MuJoCo, and has a separate AI review check it against your request.
  4. The experiment opens paused, with every assumption the AI made listed next to it.
  5. You run it, slow it down to a tenth of real speed or speed it up ten times, pause at any moment, and click any object to read its speed, height, mass and energy, with graphs over time.
  6. You ask for a change in a follow-up message. The current experiment is sent along, so you only describe what's different.

There are also six prepared examples that open without an AI request: a pendulum wave, double pendulums that start 5° apart, a spring net catching a ball, the three-body figure eight, a frictionless loop and dominoes around a bend.

How I built it

I planned it with the Devpost Learn skill pack (scope, PRD, technical spec and a build checklist, all in the repo), then built it with AI coding agents.

  • Front end: React and Three.js. MuJoCo 3.16 runs in the browser as WebAssembly, so the motion you see comes from a real physics engine. Nothing is animated by hand.
  • AI layer: the Pi agent SDK, with a small set of lab tools. The agent runs inside a disposable Docker container with no network access, read-only files, a non-root user and bounded resources. Each update is limited to 16 model requests, 24 tool calls and four minutes.
  • Models: the demo uses GPT-6 Luna through a ChatGPT subscription. OpenRouter, OpenAI and Anthropic API keys also work.
  • Hosting: Google Cloud Run for the app, Firestore for saved scenes (each scene gets its own link), and a private VM for the isolated worker.
  • Tests: 235 automated tests cover physical references, scene validation, persistence, provider boundaries and sandbox isolation.

Challenges I ran into

  • The first engine couldn't do the job. I started with Rapier, but its ropes are only maximum-distance limits: no cord routed over a pulley and no tension reading. An Atwood machine, the most basic pulley problem, was out of reach. I compared four engines with the same test problems and moved to MuJoCo, which also meant writing a compiler from the AI's scene description into MuJoCo's model format.
  • A scene that runs can still be the wrong scene. The AI tied a "spring-launched" ball to its spring, so it just bobbed in place. It built dominoes sideways, multiplied a launch speed by the projectile's mass, and kept a recoil from a shot that had been removed. That's why every build is tested in the engine, reviewed by a second AI against the request, and opens paused with its assumptions on screen.
  • The agent got stuck. In one failure it sent eight valid proposals in a row that only changed the scene's title. I separated physical checks from cosmetic ones and replaced the "submit the same thing again" handshake with an independent review of the physics.
  • Models are unreliable in different ways. Free routes appeared and disappeared, replies timed out after 60 seconds, and one in five runs of the same prompt wrongly answered "unsupported". Unsupported verdicts now get a second look, failures keep your previous experiment and your message, and every update has hard limits (16 model requests, 24 tool calls, four minutes) so nothing runs away.
  • "Fire three shots, ten seconds apart" became one shot. The model treated repeated firing as outside the physics, so I added scheduled actions (launch, release and timed loads) that the engine runs at exact simulation times.
  • Letting the AI see the scene, safely. Models that accept images review rendered frames from a sandboxed browser with no network. Getting it running meant fixing a missing writable profile directory, and a security policy that blocked the physics engine's own WebAssembly startup code.
  • Real time is hard in a browser tab. Contact needs small time steps (16 sub-steps per millisecond), so dense scenes fall behind real time on slower laptops. The app labels this as slow motion instead of skipping steps, and pauses cleanly if the tab stalls rather than jumping ahead.
  • Physical detail matters. For the demo, a domino chain had to release a Newton's cradle. The balls needed small gaps and perfect elasticity, the pin needed friction and the right thickness, and the pulled-back ball had to actually touch the pin. The AI caught two of these itself and asked me clarifying questions.

Accomplishments that I'm proud of

  • A textbook check that agrees: a 1 kg block on a 30° ramp with friction 0.2 reaches 3.60 m/s in the lab, against 3.58 m/s on paper.
  • A chain reaction built from four plain requests: a spring launcher, ten dominoes, a Newton's cradle and a release pin. In the final run the far ball swings out as far as the first ball came in, while the middle balls barely move.
  • The pendulum wave lines up again at 30 seconds to within 0.05 mm.

What I learned

  • Test the physics engine with numbers, not documentation. I ran the same checks (an Atwood pulley, a projectile, friction) on Rapier, PhysX, Jolt and MuJoCo. MuJoCo matched the textbook pulley acceleration and tension, and gave exactly the same result at 0.1×, 1× and 10× playback, which is what an inspectable lab needs.
  • "Valid" is not "correct". The AI produced scenes that passed every schema check but meant something else: a joint used as a slider, a starting displacement applied twice, a launch speed multiplied by mass. Schema descriptions now explain what each building block physically does, and the first half-second of motion is measured, not assumed.
  • A reviewer needs evidence. The separate AI review once rejected a correct ramp because a joint coordinate read zero, and another time missed that a "held" ball was a real physical attachment. Giving it measured anchors, distances and angles from the engine made its verdicts much more reliable.
  • Reference frames bite. An early velocity readout used the engine's raw internal values and reported the wrong speed for the tip of a pendulum. Readings now use the engine's world-frame velocities, and the test suite checks them against analytic results.
  • Performance is mostly drawing, until it's contacts. One scene was making 411 draw calls per frame; batching fixed that. Dense contact scenes stay expensive, and a browser Web Worker only makes them more responsive, not faster.
  • Remove what doesn't work. A force overlay I added looked unstable and confusing, so I withdrew it instead of polishing it.
  • Follow-ups need memory. After a page refresh, a follow-up request lost the original description and assumptions. Saved scenes now carry them, so "make the ramp steeper" still means the right ramp.

What's next

The open issues on the repo outline the roadmap:

  • Heavy scenes on server-side workers: run large contact scenes, like a hundred bricks, on isolated server physics workers and stream the results, since the browser can't keep up.
  • Changing systems: rockets that lose mass, bodies that break or merge, firing sequences and feedback controllers, all from declarative building blocks rather than model-written code.
  • Geometry and scale: imported collision shapes, plus validated small, fast and astronomical scales.
  • Deformable materials: bending beams, stress and strain, plasticity and fracture, so a rubber ball can really squash.
  • New areas of physics: fluids and aerodynamics, thermodynamics, electricity and magnetism, and optics and waves.
  • Editing in the inspector: change an object's properties directly instead of only reading them.

Built With

Share this project:

Updates

Submission history