Projector Space

• 

• 

• 

• 

Inspiration

Projective texturing is visually powerful, but most examples stop at a shader demo or a video mapped onto a single plane. I wanted to build something that could work inside a real Three.js application: one HTML video projected across multiple selected objects, with reliable playback, receiver updates, cleanup, authoring, persistence, and public presentation.

I deliberately started inside Garden Planner, an existing Three.js editor, instead of beginning with an isolated sandbox. The application provided real constraints: dynamically rebuilt geometry, editor camera interactions, autosave, publishing, autoplay restrictions, media replacement, and long-lived runtime resources.

What it does

Projector Space is a host-neutral Three.js runtime for projecting real HTML video across host-selected 3D geometry.

The host explicitly registers receiver meshes or receiver roots. The package then manages:

• the HTML video element and VideoTexture;
• the projector camera and projector-space mapping;
• shader overlays for registered receivers;
• playback state and media status;
• receiver refresh and subtree detachment;
• transactional construction and best-effort disposal.

Projector pose is independent from receiver selection. A host can move or aim the projector without rebuilding the media session, and it can rebuild receiver geometry without restarting playback.

How I built it

Garden Planner existed before Build Week. During the event I built the projective-media feature, world-space projector authoring, persistence and public-read integration, the reusable package, the standalone sandbox, tests, CI, and the external dependency cutover.

The implementation followed three stages:

1. Application-scale integration
   The runtime was first developed inside Garden Planner, where lifecycle, ownership, persistence, input, and autoplay problems could be observed under real application conditions.

2. Boundary extraction
   Product-owned concerns stayed in Garden Planner: UI, authored state, media catalog, asset resolution, publishing, and receiver policy. Reusable Three.js and browser-media mechanics were extracted into three-projective-media.

3. Independent validation
   A standalone sandbox was built using only the public package entry point. Garden Planner was then switched to consume the package as a pinned external dependency.

Challenges

The hardest part was not the projection shader itself. The main challenges were resource ownership and lifecycle correctness.

Receiver objects can be rebuilt or removed while a video is playing. Media can be replaced while projector pose and receiver policy must remain stable. Construction can fail after only some resources have been created. Disposal must continue even if one callback throws.

Another challenge was separating product behavior from reusable runtime behavior. Garden Planner understands buildings, authored projects, media catalogs, public scenes, and editor tools. The package must understand none of those concepts.

The project also had to support muted public playback, browser autoplay restrictions, saved projector pose, public read-only controls, and migration of earlier persisted projector records.

Accomplishments

• Extracted a reusable, host-neutral Three.js package.
• Preserved explicit receiver ownership without global scene scanning.
• Supported media replacement without corrupting projector pose or receiver registration.
• Supported receiver rebuilds without restarting the active playback session.
• Added transactional construction and idempotent, best-effort cleanup.
• Created a standalone public-API sandbox with multiple video sources.
• Reconnected Garden Planner through the external package dependency.
• Published Nocturne Garden as a live application-scale reference integration.
• Added package tests, CI, GitHub Pages deployment, and public validation documentation.

What I learned

A small sandbox is useful for portability and API documentation, but it is not always the best place to discover a reusable boundary.

Starting inside a mature application exposed problems that would have remained invisible in a static shader demo. Only after those problems were understood was it possible to extract a smaller and more reliable package.

I also learned that projector pose, media ownership, and receiver lifecycle should remain independent concerns. That separation made the final API smaller and made both the sandbox and Garden Planner integration easier to reason about.

Development workflow

I used Codex for implementation, focused tests, browser validation, extraction, and the dependency cutover. GPT-5.6 supported architecture analysis, failure-mode review, and evaluation of the package boundary. Product, UX, and final architecture decisions remained mine.

What's next

The current package is a pre-release distributed from its public repository.

Possible next steps include:

• npm distribution after a separate release review;
• additional browser validation;
• improved authoring helpers;
• projector occlusion and depth handling;
• keystone correction;
• multi-projector blending;
• broader receiver support.

Links

• Public repository
• Standalone sandbox
• Nocturne Garden reference integration

Built With

• codex
• glsl
• gpt-5.6
• html5
• javascript
• node.js
• npm
• three.js
• video
• vite