Physical AI Lab

Inspiration

Physical AI is becoming important far beyond robotics specialists, but many learning materials jump from vocabulary to mathematics before a newcomer has physical intuition. We wanted concepts such as latency, friction, uncertainty, calibration, and safety margins to feel concrete: if you move a parameter, you should see what changes in the world and understand why.

What it does

Physical AI Lab is a bilingual interactive course with 26 browser-based experiments. Instead of presenting sensing, feedback control, Sim-to-Real, and robot safety as abstract equations, each mission starts with a deliberately weak baseline. Learners observe a visible physical failure, inspect the measured state, change one causal parameter, and verify the result against an acceptance envelope.

The Build Week judge route highlights three missions:

• M20 — Reality Gap: sunlight, slippery packaging, and camera drift become measurable commissioning residuals.
• M22 — Runtime Safety: missing depth, stale images, sensor bias, and a stuck stop command are injected while a cobot carries an 18 kg beverage case near a worker.
• M26 — Warehouse Capstone: perception, braking, traction, grasp force, and re-sensing are combined across three different orders.

Every experiment uses the same loop: Predict → Observe → Change → Verify. Failures are recorded as evidence; success is derived from the same state shown by the meters and animation.

How we built it

The product is a self-contained HTML simulation with an embedded Three.js engine, wrapped by a small Vinext application for OpenAI Sites. It has no runtime API dependency. Simulation state, animation, meters, logs, and pass/fail gates share one source of truth. Progress and experiment records stay in local browser storage.

Codex and GPT-5.6 were implementation and verification partners. They helped convert qualitative product feedback into acceptance tests, implement the experimenter loop and 3D scenes, add the bilingual judge route, write regression tests, and prepare deployment and submission artifacts. The human product owner repeatedly tested the experience and redirected it whenever causality or the real-world use case was unclear.

Challenges we ran into

The hardest problem was not drawing a 3D robot; it was making the animation, meter, explanation, and acceptance decision tell exactly the same physical story. Early versions were too abstract and text-heavy. We redesigned difficult missions around recognizable logistics and safety jobs, introduced input-to-physical-effect boards, and made the first failed baseline part of the teaching loop.

Accomplishments that we're proud of

• 26 working experiments in one offline-capable artifact
• A five-minute English judge route without removing the full Japanese curriculum
• Visible causal links between parameter changes and physical outcomes
• Safety gates based on measured separation, force, latency, friction, and uncertainty—not cosmetic completion states
• Automated regression coverage plus independent desktop/mobile acceptance QA

What we learned

For physical AI education, a failure can be more informative than a perfect demo if the learner can see the violated margin and test one causal correction. A digital twin becomes understandable when its residual is tied to a specific real-world variation, not when it is presented as a generic twin diagram.

What's next

• Finish English coverage for optional knowledge gates and low-level labels
• Add instructor-authored experiment sets and shareable progress records
• Validate the simulations with robotics educators and real commissioning data
• Add accessible narrated walkthroughs and richer keyboard guidance

Built With

• codex
• gpt-5.6
• javascript
• openai-sites
• react
• three.js
• typescript
• vinext