TripoSplat for WebGPU and Mac

CUDA-era AI 3D generation, ported by Codex GPT 5.6 Sol, to WebGPU running locally install-free in the browser - and as native Apple Silicon app!

Comment

TripoSplat WebGPU + Mac

Inspiration

TripoSplat can generate a 3D Gaussian splat from a single image, but its original research implementation belongs to a CUDA-oriented ecosystem. Running models like this usually requires an NVIDIA GPU, Python dependencies, model-specific setup, and substantial technical knowledge.

I wanted to test a more ambitious use of coding agents:

Could Codex GPT-5.6 Sol take a research-grade CUDA-era AI 3D pipeline, understand it deeply enough to port it, and turn it into software that runs locally on hardware the original implementation did not target?

The first goal was an install-free WebGPU implementation that could run directly in the browser, without sending the source image to an inference server.

During optimization, Codex unexpectedly developed a Mac-specific execution path. That divergence revealed a second useful product: a native Apple Silicon application derived from the WebGPU port.

The result is one project with two complementary runtimes:

  • TripoSplat WebGPU, for local, install-free generation in a WebGPU-capable browser
  • TripoSplatMac, a native application designed specifically for Apple Silicon

What It Does

TripoSplat WebGPU takes a single image and reconstructs it as a 3D Gaussian splat locally in the browser.

There is:

  • No CUDA requirement
  • No Python installation
  • No remote inference server
  • No model upload to the cloud
  • No native installation for the WebGPU version

The generated splat can then be viewed interactively in 3D.

TripoSplatMac provides a native Apple Silicon path for users who prefer a dedicated Mac application.

How We Used Codex GPT-5.6 Sol

Codex was not used only to generate isolated functions or boilerplate. It was used as an agentic engineering partner across the entire porting process.

Codex helped:

  • Inspect and understand the original research code
  • Trace the model architecture and inference pipeline
  • Identify assumptions tied to CUDA, PyTorch, and the original runtime
  • Design a browser-compatible execution architecture
  • Translate GPU operations into WebGPU-compatible implementations
  • Replace or restructure unsupported operations
  • Generate and debug WGSL compute shaders
  • Diagnose shader validation and numerical failures
  • Build the browser interface and interactive 3D viewer
  • Create test and qualification tooling
  • Improve memory use and execution behavior
  • Document the resulting implementation
  • Derive the native Mac execution path from the WebGPU work

Kiro with GPT-5.6 Sol Max was also used as a complementary coding environment during implementation and refinement.

The human role was primarily to define the objective, inspect results, identify failures, challenge incorrect approaches, and steer the agents toward a working implementation.

Agentic Development Process

The development process followed a repeated engineering loop:

  1. Inspect the existing implementation
  2. Form an architectural plan
  3. Implement a candidate WebGPU path
  4. Run the application
  5. observe compilation, validation, numerical, or output failures
  6. Trace the failure across the pipeline
  7. Revise the shaders, buffers, tensor operations, or orchestration
  8. Re-run qualification
  9. Compare output and performance
  10. Document the working path

This matters because porting GPU software is not a simple language translation task. The agent had to reason across model architecture, tensor operations, memory layouts, browser GPU constraints, shader behavior, rendering, and application integration.

Technical Implementation

Model and Runtime Translation

The original implementation depended on assumptions from a CUDA and PyTorch-oriented runtime.

The WebGPU port replaces those assumptions with browser-compatible components, including:

  • WGSL compute shaders
  • WebGPU storage and uniform buffers
  • Explicit bind groups and pipeline layouts
  • Browser-compatible tensor and math operations
  • JavaScript and TypeScript orchestration
  • Local model loading and inference
  • GPU-to-renderer data transfer

Unsupported operations could not always be translated directly. Some required alternative implementations, decomposition into smaller kernels, or changes to the surrounding execution strategy.

Gaussian Splat Pipeline

The application converts the model output into data suitable for interactive Gaussian splat rendering.

The pipeline includes:

  • Model input preparation
  • GPU inference
  • Splat attribute generation
  • Splat data packing
  • Camera-space projection
  • Interactive rendering
  • Camera and viewport controls

Local Browser Execution

The WebGPU version performs inference on the user’s own compatible GPU.

The browser acts as the application runtime rather than as a thin interface to a hosted backend. This reduces deployment friction, avoids per-generation server cost, keeps source images local, and makes offline execution possible once the required assets are available.

Try it out without install: https://triposplat-webgpu.vercel.app/

Native Apple Silicon Path

TripoSplatMac was not part of the original plan.

While improving the WebGPU implementation, Codex created a Mac-specific path. Instead of discarding it as an accidental divergence, I recognized that it could support a distinct native experience optimized around Apple Silicon.

This became TripoSplatMac. It works with ANY Mac Silicon MacOS 14 and higher. (No need to upgrade your MacOS!)

The two versions now demonstrate different forms of accessibility:

  • WebGPU broadens access through the browser
  • TripoSplatMac provides a native path for Apple Silicon users

Try it out by grabbing the dmg: https://github.com/ai3d-dev/TripoSplatMacApp

Challenges

We didn't really sleep all week.

Translating a Research Implementation

Research repositories are rarely structured as portable products. Codex had to trace dependencies, implicit assumptions, intermediate representations, and model-specific behavior before it could construct a viable alternative runtime.

Replacing CUDA-Oriented Operations

WebGPU and WGSL do not provide direct equivalents for every operation used in PyTorch or CUDA environments. Some operations required custom kernels, decomposition, altered memory layouts, or new execution strategies.

Debugging Across Multiple Layers

A visually incorrect result could originate from:

  • Input preprocessing
  • Tensor shape assumptions
  • Buffer offsets
  • Numeric precision
  • Shader logic
  • Workgroup sizing
  • Resource binding
  • Camera projection
  • Splat packing
  • Rendering

Codex had to reason across these layers rather than treating each failure as an isolated syntax error.

WebGPU Validation

WebGPU is deliberately strict about buffer usage, alignment, shader types, bindings, and pipeline construction. The implementation required repeated validation and debugging cycles before the full path could run coherently.

Productizing the Port

A successful kernel port alone would not make a useful project. The work also required local model loading, progress and error handling, a viewer, camera controls, browser capability detection, documentation, and a runnable interface.

Accomplishments

  • Ported a CUDA-era AI 3D generation workflow to WebGPU
  • Ran model inference locally in a WebGPU-capable browser
  • Removed the requirement for Python, CUDA, and a remote inference server
  • Built an install-free image-to-3D experience
  • Created an interactive Gaussian splat viewer
  • Developed qualification and debugging tools for the port
  • Derived a separate native Apple Silicon application from the WebGPU work
  • Used Codex GPT-5.6 Sol as an agentic engineering partner across architecture, implementation, debugging, and refinement

What I Learned

The most interesting result was not simply that Codex could write WGSL.

The project showed that GPT-5.6 Sol could work across an unusually wide technical surface: research code, model architecture, GPU compute, shader compilation, browser APIs, rendering, debugging, application design, and native platform divergence.

It also produced a product insight I had not requested. The accidental Mac-specific optimization initially appeared to be a mistake, but it exposed a useful second runtime and became TripoSplatMac.

That is a different kind of value from code completion. The agent did not merely accelerate a predetermined implementation. Its exploration changed the shape of the resulting product.

What’s Next

  • Continue improving model-output fidelity
  • Expand browser and GPU compatibility testing
  • Benchmark supported hardware transparently
  • Improve memory usage on constrained devices
  • Add richer splat inspection and editing
  • Add export and interoperability options
  • Explore multi-view reconstruction
  • Continue developing TripoSplatMac as a native Apple Silicon experience
  • Use the port as a foundation for our sister project MutualGPU, where compatible computers can contribute useful units of 3D generation to people without sufficient local compute

Final Summary

TripoSplat WebGPU + Mac brings CUDA-era AI 3D generation beyond its original hardware and software environment.

Using Codex GPT-5.6 Sol as an agentic engineering partner, the project transformed a research-oriented TripoSplat pipeline into two local runtimes: an install-free WebGPU browser application and a native Apple Silicon app.

The project demonstrates something more substantial than AI-generated code. It shows an agent reasoning across an unfamiliar GPU system, developing an alternative architecture, debugging the resulting implementation, and unexpectedly revealing a second viable product path.

CUDA-era AI 3D generation, ported by Codex GPT-5.6 Sol to WebGPU for local, install-free browser inference and to a native Apple Silicon app.

Built With

  • codex
  • gpt5.6
  • triposplat
  • webgpu
Share this project:

Updates