BlendLink XR

Inspiration

4.2 million people use Blender every month to create 3D art — all by staring at a flat screen and dragging a mouse. There's always a layer of glass between the artist and their work.

Then we held the MX Ink. A pen that knows exactly where its tip is in 3D space, how hard you're pressing, and which way it's pointing. The same natural instrument artists already know how to use — except it works in mid-air.

Nobody had connected it to professional 3D software. Blender has a fully open architecture and 4.2 million monthly users. The MX Ink has sub-millimeter tracking and analog pressure. The bridge between them didn't exist. So we built it.

What it does

BlendLink XR is a system that lets artists use the Logitech MX Ink stylus for 3D creation in Blender through VR. The project has two parts:

1. Blender C++ OpenXR patch — Makes Blender recognize and handle the MX Ink as an input device without crashing.

2. MX Ink input architecture — A complete control mapping that solves the fundamental UX problem: the MX Ink has no thumbstick, no traditional trigger, and a dock sensor. Every creative action needs exactly one gesture, and every gesture needs exactly one action.

The control scheme we designed:

• Middle cluster force → pressure-sensitive drawing (analog, like pressing a real pen)
• Front click → grab/navigate objects
• Back click → radial tool menu
• Front double-tap → undo
• Back double-tap → redo
• Tip force → stroke pressure channel (surface contact)
• Dock state → pause all input (pen put down)
• Back hold + middle force → cursor/brush resize (replaces missing thumbstick)
• Front hold + middle force → keyframe nav / proportional edit radius

For the hackathon demo, we showed this running inside Blender VR by extending an existing open-source VR addon (Freebird XR by cmdr2, GPL-2.0) with our MX Ink layer. This let us demonstrate the pen working with real tools (drawing, editing, navigating) without building an entire VR framework from scratch in hackathon time.

The demo also includes:

• Grease Pencil v3 drawing with 7 brush presets (Pencil, Ink, Marker, Airbrush, Dot)
• HSV color wheel for VR
• A glossy 3D model of the MX Ink rendered in-headset with custom Blinn-Phong shader
• Pressure-sensitive stroke width

How we built it

The Blender patch (C++): We modified GHOST_XrSession.cc in Blender's source to handle the XR_LOGITECH_mx_ink_stylus_interaction profile. Without this, Blender crashes when OpenXR reports an unsupported interaction profile. The patch gracefully registers the MX Ink and maps its components to Blender's action system.

The MX Ink control architecture (Python): We designed and implemented:

• OpenXR action bindings for every MX Ink component (tip force, tip pose, cluster buttons, dock state, double-taps)
• A combo system that replaces the missing thumbstick: modifier buttons + middle force for cursor resize, frame navigation, and proportional edit control
• Dock state tracking that suppresses input when the pen is holstered
• Automatic fallback to Touch controllers when MX Ink isn't connected

Grease Pencil brush system (Python):

• Seven brush presets with distinct pressure curves, opacity, jitter, and softness
• Native Blender GP v3 API integration (drawing.add_strokes(), per-point radius/opacity/color)
• HSV color wheel rendered with a custom GPU shader

MX Ink 3D model (Python + shader):

• Exported from the official FBX, triangulated, normals computed
• Custom Blinn-Phong shader for glossy appearance with specular highlights
• Tip aligned to cursor position in real-time

Demo integration: For the visual demo, we extended Freebird XR (a commercial Blender VR addon by cmdr2) with our MX Ink layer so people could see the pen working with a full VR toolset. The VR framework, existing tools (select, erase, shapes, mesh editing), navigation, and UI system are cmdr2's work — not ours.

Challenges we ran into

The MX Ink has no thumbstick. This is the single biggest UX challenge. Standard VR controllers use the thumbstick for undo/redo, cursor resize, and frame navigation. We solved it with a combo system that repurposes button hold + middle force for these functions.

Trigger mapping was wrong on first try. We initially mapped tip force to trigger. It worked for clicking buttons but not for drawing — tip force only activates on surface contact, not mid-air. We had to completely rethink: middle cluster force became the drawing trigger, front click became grab, tip force became a separate pressure channel.

GP materials in Blender 5.x. Regular materials don't render as GP strokes — they show up white. GP materials require special creation through bpy.ops.object.grease_pencil_add(). Took hours to debug.

Analog pressure mapping. The middle cluster force is 0-1 analog. Mapping this to natural-feeling stroke width required per-brush pressure curves — light force for thin lines, heavy force for thick. Each of the 7 brushes has a different response curve.

Accomplishments that we're proud of

The control mapping feels natural. After three complete rewrites, artists pick up the MX Ink and start drawing without instructions. The gestures map to what your hand wants to do.

Seven brush presets that feel different. Not just "thin line, thick line" — each brush has distinct character. Pencil is scratchy and responsive. Airbrush is soft and gradual. Marker is bold and flat.

The 3D pen model in VR. Custom Blinn-Phong shader, specular highlights, dark charcoal finish. It looks like the real pen. The tip aligns with the cursor to the millimeter.

It actually draws real Blender data. Take the headset off and every stroke, every edit, every object is right there on the desktop. No export, no conversion.

What we learned

The MX Ink isn't a controller with extra buttons. It's a fundamentally different input device that requires a fundamentally different control architecture. You can't map it to existing controller bindings. The "no thumbstick" constraint forces creative solutions that end up feeling more natural than the original.

The hardest bugs are the silent ones. Strokes render white (material issue). The pen points backward (axis conversion). The trigger fires but nothing draws (wrong force channel). Understanding the full pipeline from hardware to pixels is the only way to debug these.

What's next for BlendLink XR

Blender upstream contribution. The MX Ink OpenXR patch is being prepared for Blender's official codebase so all Blender VR users get MX Ink support natively.

Custom XR framework. Building our own lightweight spatial UI layer from scratch, purpose-built for stylus input rather than adapted from controller-based designs.

Horizon Link companion app. A dedicated Meta Quest application that captures MX Ink input at full fidelity and connects to Blender over a low-latency localhost bridge at 72-120Hz.

Sculpt mode. Push, pull, smooth, and carve mesh surfaces with real pen pressure.

---

Attribution

The hackathon demo used Freebird XR by cmdr2 as the VR base layer. Freebird XR is a commercial Blender VR addon — please purchase it if you want VR creation in Blender.

Our original work: MX Ink OpenXR patch, control architecture, combo system, GP brush presets, color wheel, and pen model shader.

Built With

• c++
• openxr
• python
• unity