Inspiration

On a Shinkansen, the tray table is about 41 × 26 cm — too small for a laptop, exactly right for your hands. Why carry screens at all, when every seat already has a surface? JIMEN turns any real table into a touch panel with just passthrough and hand tracking, borrowing desktop OS grammar — desktops, dock, windows — in a medium impossible anywhere else: content lives flat on the surface and pops out of a hole in the desk.

What it does

Touch to boot. Place both palms on the table and JIMEN wakes up: the edges light up and app icons spread out. Run a fingertip across them and the one under your finger stands up to face you — the table is the touch panel.

Desks, not screens. Content lives on switchable "desks." Slide the whole table surface to flip between them, each keeping its own state.

Three apps, one loop. Block: an art tool, not a game — touch a cell on the desk grid and it rises one layer; sculpt shapes with your fingertips, then dive into your miniature at full scale. Creations publish to SceneSync, the author's own scene-sync service, so a piece can be reopened and co-edited from other engines and the web. Air hockey: your finger is the mallet, against an AI opponent, in 3D. Work: a real web browser floats above the desk, operated with the on-desk keyboard and touchpad — far things are controlled through the table, never by ray.

A hole in the desk. One hole is a window into the world below, a port things pop out of, and the trash. Designed for the tray table first, everything playable seated, hands-only, in under ten minutes.

How we built it

Unity with the Meta XR SDK (v207), targeting Quest with full hand tracking — no controllers anywhere. Quest has no ARKit-style dynamic plane detection, so we built our own in three layers: Environment Depth fits the plane, the Passthrough Camera API finds the outline, and a real fingertip touch confirms the height. Setup is one gesture: put your palm down. Touch uses fingertip-to-plane distance with hysteresis, plus a ripple and sound on contact — the surface itself absorbs hand tremor, unlike ray pointing.

Challenges we ran into

The hard part was detection on small, glossy tray surfaces, and keeping touch accurate during fast air-hockey play. On a real Quest: 120 Hz mode implemented, and the shipped real-hands occlusion runs at about 120 fps. On a real Shinkansen tray table the build felt workable; we are still confirming how surface content holds before claiming train use.

Accomplishments that we're proud of

The moment a plain tray table wakes up under your palm — touch confirmation no ray or floating UI can give, inside the two-foot radius of an airplane seat.

What we learned

Familiar grammar, impossible medium. Borrowing desktop conventions made a new input surface learnable in seconds, so all the surprise budget could go into playfulness.

What's next for JIMEN

More desks and apps: a fishing pond in the hole whose caught fish swim your room, a music player that expands into a room-filling spatial VJ, richer AI Work integrations, and polish for a public release.

Built With

  • environment-depth-api
  • meta-xr-sdk
  • passthrough-camera-api
  • scenesync
  • unity
  • xr-hands
Share this project:

Updates

Submission history