-
-
thumbnail
Inspiration
Quest developers constantly wait on Android builds just to test small changes to XR scenes and interactions. MetalQuestLink uses the Mac's Metal pipeline to bring that feedback loop into Unity Editor Play Mode on Apple Silicon.
What it does
MetalQuestLink intercepts Unity's OpenXR frames through an implicit API layer, encodes the stereo images with VideoToolbox, and streams them to a prebuilt Quest client. The headset submits the received video as a stereo projection layer and sends HMD, Touch controller, and hand-tracking input back to the Editor. It also forwards haptics and provides a documented, approximate passthrough preview.
How it was built
The macOS side is a C++/Objective-C++ OpenXR API layer built on Metal and VideoToolbox. A binary protocol carries video metadata, poses, controller state, 26 joints per hand, clock-sync data, and haptic commands. The Quest client is built with Unity, MediaCodec, and a native OpenXR projection extension. A self-contained UPM package bundles the native layer and the APK, configures Standalone OpenXR, runs preflight checks, and provides a one-click setup flow.
How Codex and GPT-5.6 were used
Codex with GPT-5.6 was used throughout development: to understand the existing native and Unity integration, trace version-coupled assumptions, implement the compatibility matrix and quick-setup flow, write regression tests, and keep the setup, specification, open-source-contribution, and submission docs consistent with one another. The native, Unity, Simulator, and Quest checks remained the source of truth.
Challenges and accomplishments
The two devices use different monotonic clock epochs; OpenXR projects vary in loader order and swapchain layout; and a low-latency path has to avoid building up encoder backlog. MetalQuestLink addresses these with ping/pong clock estimation, non-destructive loader ordering, support for both array and per-eye swapchains, and a bounded frame queue that reports dropped frames.
The release includes a prebuilt UPM tarball and Quest APK, stereo compositor projection, bidirectional HMD/controller/hand/haptic transport, checksums, a doctor command, a clean-room end-to-end tarball test, and a multi-version Unity package matrix. Quest 3 end-to-end tests recorded up to 74 fps received, 76 fps decoded, and pose sampling at 73 Hz.
What's next
Expand the verified Unity matrix, add CI-backed package tests, improve notarized distribution, measure optical latency and quality over long sessions, and grow the project through community issues and pull requests.
