NibGlide

Inspiration

Pen displays make drawing feel direct, but navigating macOS still repeatedly pulls the user's hand away from the pen. Scrolling a brush palette, moving through a reference page, or going back in a browser often means reaching for a trackpad or mouse. Existing remapping approaches can generate scroll events, but the difficult part is deciding when the user means to draw, click, select text, or scroll—and doing it without slowing every pointer event on the Mac.

I wanted scrolling to feel like a quiet extension of the pen: direct while touching, physical after release, predictable in browsers, and completely absent when the user is drawing.

What it does

The app runs in the macOS menu bar and transforms only tablet-marked pen contact.

• Direct pen movement scrolls the content under the cursor.
• Release velocity produces controlled exponential momentum.
• Automatic mode preserves a slightly imprecise short tap, turns slow movement from whitespace into scrolling, and hands a deliberate hold-and-drag back to macOS for live text selection or object dragging.
• A subtle white-and-gray arc beside the pointer fills during the roughly 0.20-second hold, making the moment native dragging becomes ready visible instead of forcing the user to guess.
• In Chrome and other browsers, only positively identified text, links, or images can become a native drag. Web whitespace and unresolved contacts remain pen-scroll surfaces, avoiding the surprising text selection that a mouse drag can start from blank space.
• During momentum, a short recontact acts as a brake without leaking a click or image drag; continued movement becomes a new scroll.
• Automatic mode protects drawing applications and leaves the pen native by default.
• Holding a configurable key temporarily creates a dedicated scroll contact, preventing accidental brush selection in dense palettes.
• Horizontal pen gestures in browser content preview Back or Forward with an edge-clipped affordance. Fast, slightly short gestures can commit through velocity projection, while pulling back cancels them.
• A bounded fail-open recovery system ensures that an input-engine fault never traps the mouse, keyboard, trackpad, or pen.
• Onboarding completion and runtime health are separate, so a recoverable fault never turns into another unwanted welcome screen.

How it was built

The project is a Swift 6 macOS 15+ application using AppKit and public system frameworks. CoreGraphics event taps provide the input boundary. A dedicated serial input thread owns all gesture and recovery state so high-frequency input never waits for Accessibility calls, window enumeration, logging, or the main UI thread.

The pure core module contains the scroll state machine, velocity estimator, momentum integrator, browser-history recognizer, interaction policy, settings migration, and fail-open recovery policy. AppKit code handles the menu-bar experience, onboarding, settings, browser overlay, and optional Evidence Sessions.

At contact start, NibGlide performs at most one bounded asynchronous Accessibility lookup. The result informs a contact-local arbitration state machine rather than adding work to hover or every move packet. A short contact can remain a click despite natural landing jitter; movement can become direct scrolling; and a stationary contact over a confirmed selectable target can complete the hold meter and return to native macOS dragging. Narrow caret positions at line starts, between characters, and at line ends are included so selection begins where people actually place the pen.

The browser interaction combines this hold-to-select path with a short direction-lock hysteresis, an 80-point normal history-commit threshold, velocity-based endpoint projection for imperfect fast releases, and a pull-back cancellation threshold. Prompt horizontal movement remains Back or Forward; completing the hold permanently chooses native selection for that contact. The visual history affordance stops before its final position until the recognizer is genuinely armed.

How Codex and GPT-5.6 were used

Codex with GPT-5.6 was the primary development collaborator across research, architecture, implementation, testing, and evidence analysis.

An early implementation made the cursor extremely heavy even when scrolling was disabled. Codex helped inspect the complete pointer path, compare it with low-latency precedents such as Karabiner-Elements, and redesign the app so the input callback performs only bounded state decisions. Expensive Accessibility and window work moved off the hover path, and transient tap failures became fail-open health states rather than onboarding resets.

Later, Clip Studio Paint frequently ignored momentum. We added narrowly scoped Evidence Session diagnostics and analyzed a real session. All 31 tested contacts passed the momentum decision and started the timer, but the first frame in every sequence was only 0.04–0.29 pixels and rounded to a zero integer wheel delta. We fixed the phase protocol by accumulating subpixel motion and posting .began only when the first nonzero wheel displacement exists.

For browser history, Codex researched Apple's fluid-interface guidance, Microsoft's cross-slide state model, Chromium, and WebKit. We converted that research and the user's failed 76–92 point gestures into velocity projection, hysteresis, cancellation tolerance, and a visible speed bump. Automated tests cover fast-short acceptance, slow-short rejection, pull-back cancellation, vertical rejection, and subpixel momentum accumulation.

The final automatic-mode work focused on human uncertainty rather than requiring pixel-perfect pen control. Real-device sessions exposed ambiguous landings, accidental drags while braking momentum, slow whitespace swipes, and competition between browser history and text selection. Codex helped turn those observations into explicit click, brake, scroll, history, and delayed-native-drag states, plus a restrained progress cue that communicates when a hold has committed.

The project emerged from repeated real-device loops: build, use, record evidence, inspect the code and external research, revise, and test again.

Challenges

• Preserving native drawing while offering scrolling inside the same creative application.
• Preventing a held activation key from leaking clicks into a Clip Studio brush grid.
• Preserving ordinary clicks and live text selection without letting blank browser areas leak into strange native mouse-drag selection.
• Resolving the conflict between an immediate horizontal browser-history gesture and a deliberate horizontal text selection from the same starting point.
• Keeping a global event tap responsive enough that merely launching the app has no effect on cursor latency.
• Producing browser-history feedback without private trackpad APIs or pretending synthesized events are physical multitouch.
• Distinguishing onboarding from recoverable permission and event-tap health.
• Diagnosing an application-specific momentum failure whose launch decision was actually correct.

Accomplishments

• The app remains idle and lightweight until relevant pen contact arrives.
• Mouse, trackpad, and unrelated keyboard input remain outside the conversion path.
• Drawing applications default to native pen behavior, with an explicit temporary scroll override.
• Automatic mode tolerates natural tap jitter, supports slow whitespace scrolling, brakes momentum safely, and makes delayed native selection visible.
• Text selection works from realistic caret positions across native apps, while browser whitespace remains a reliable scroll surface.
• Browser Back and Forward have a coherent direct-manipulation preview aligned to the real web-content bounds, including layouts with vertical tabs.
• Clip Studio receives a valid nonzero momentum begin sequence.
• Development diagnostics remain hidden from the normal product UI but can produce reproducible Evidence Sessions when needed.
• The core behavior is covered by deterministic release tests and a detailed manual hardware matrix.

What I learned

Input utilities are defined more by what they refuse to do than by the events they synthesize. A correct scroll delta is not enough: callback latency, event phase ordering, focus, click leakage, drawing intent, recovery behavior, and visual commitment all shape whether the tool feels native.

Evidence also changed the direction of development. The most useful fix for Clip Studio was not a larger velocity multiplier or a new timer; it was recognizing that a zero-distance begin frame invalidated the semantic sequence for one consumer.

What's next

• Validate additional pen-display vendors and driver versions.
• Expand hardware compatibility beyond the current Huion-focused boundary.
• Add a privacy-preserving user-facing diagnostic report separate from developer Evidence Sessions.
• Continue tuning momentum and gesture thresholds from opt-in real-device evidence.
• Explore per-application interaction profiles only where they reduce complexity instead of creating a settings burden.

Built With

• accessibility
• appkit
• codex
• coregraphics
• gpt-5.6
• screencapturekit
• servicemanagement
• swift