OCML GPU Interaction Engine

An adaptive experience engine built with React, Three.js, GPT-5.6, and Codex that turns user input into real-time 3D motion, shader effects, and performance-aware brand interactions.

Comment

Inspiration

Most interactive brand websites are built as one-off experiences. The visual result may be impressive, but the underlying interaction logic is often tightly coupled to a single page, difficult to reuse, and expensive to maintain.

I wanted to explore a different approach: build the OC Media Lab "under construction" experience as the first implementation of a reusable real-time interaction engine rather than treating it as a disposable landing page.

The goal became the OCML WebGPU Experience Engine — an experimental frontend system for translating human input into responsive 3D motion, materials, lighting, particles, and camera behavior.

What it does

OCML WebGPU Experience Engine powers an interactive 3D brand experience directly in the browser.

The system is designed around real-time input and rendering signals such as:

  • pointer position
  • pointer velocity
  • user interaction
  • animation timing
  • viewport and device characteristics
  • rendering performance

These signals can drive visual systems including:

  • 3D object movement
  • material and shader effects
  • lighting responses
  • particle behavior
  • camera motion
  • animation intensity

The OC Media Lab under-construction page serves as the first real-world implementation of the engine.

Rather than showing visitors a traditional static "coming soon" screen, the page turns construction itself into an interactive digital experience.

How I built it

The project is built with a modern React and real-time graphics stack centered around:

  • React
  • TypeScript
  • Vite
  • Three.js
  • React Three Fiber
  • WebGPU-oriented rendering
  • GSAP
  • custom real-time graphics and interaction logic

I used Codex with GPT-5.6 as an engineering partner during the hackathon development process.

Instead of asking Codex to generate an entire application from a single prompt, I used it as part of a structured software-engineering workflow:

  1. Inspect the existing repository and architecture.
  2. Identify the rendering and interaction systems.
  3. Create an implementation plan before modifying core graphics code.
  4. Implement a focused new interaction layer.
  5. Review the generated changes and architectural decisions.
  6. Build and validate the production application.
  7. Document which functionality existed before the hackathon and which functionality was added during the Codex-assisted build.

This allowed me to use Codex for meaningful engineering work while retaining human control over the visual direction, architecture, performance decisions, and final implementation.

Challenges

One of the largest challenges with highly interactive 3D websites is balancing visual complexity with runtime performance.

Real-time graphics code can easily introduce unnecessary allocations, excessive React state updates, expensive shader work, or rendering behavior that performs well on a desktop GPU but poorly on lower-powered hardware.

Another challenge was designing interaction behavior that feels intentional rather than simply adding visual effects for their own sake.

The interaction system therefore needed to remain responsive while keeping high-frequency rendering work separate from unnecessary React updates.

A further challenge was extending an existing OC Media Lab project while maintaining a clear distinction between the original application and the new functionality created during OpenAI Build Week.

Accomplishments that I'm proud of

I am especially proud of turning what began as an under-construction page into the foundation for a reusable interactive graphics system.

The project combines frontend engineering, interaction design, motion design, and real-time 3D rendering rather than treating those disciplines as separate layers.

I am also proud of using Codex as part of a controlled engineering workflow instead of treating AI-generated code as a black box. The process includes repository-level instructions, implementation planning, validation, Git history, and documentation of both AI contributions and human engineering decisions.

What I learned

This project reinforced that AI coding tools are most powerful when they operate inside a well-defined engineering environment.

Clear repository instructions, architectural constraints, validation requirements, and small measurable goals produced much better results than unrestricted code generation.

I also gained a deeper understanding of how interaction data can be treated as a reusable input layer for real-time 3D experiences rather than being embedded directly into individual components.

What's next

The next step is expanding OCML WebGPU Experience Engine beyond this first implementation.

Future versions could provide reusable modules for:

  • adaptive rendering quality
  • reusable shader interaction signals
  • configurable particle systems
  • camera behavior
  • motion accessibility controls
  • WebGPU capability detection
  • visual debugging and performance metrics
  • configurable interaction presets

The long-term goal is to make sophisticated interactive brand experiences easier to prototype, reuse, optimize, and deploy without rebuilding the entire interaction architecture for every project.

Built With

Share this project:

Updates