Cursivis Next

The most frustrating part of using AI is often not getting the answer. It is stopping your work to explain everything the AI could already see.

Inspiration

Artificial intelligence has become dramatically more capable, but the way we interact with it still feels disconnected from real work.

When we encounter a difficult paragraph, an unfamiliar word, a piece of code, an email, a form, or an image, the usual AI workflow looks like this:

Copy the content.
Leave the application.
Open a chatbot.
Paste the content.
Explain where it came from.
Describe what we want.
Wait for a response.
Copy the answer back.

The intelligence may be powerful, but the interaction repeatedly breaks concentration.

Real work does not begin inside a blank prompt box. It begins inside a browser tab, code editor, document, email thread, form, or image already open on the screen.

That led us to a simple question:

What if AI could understand the context that is already in front of the user?

Cursivis Next was built around that idea.

Its interaction model is:

$$ \text{Selection} + \text{Shortcut} + \text{Context} \rightarrow \text{Understanding} \rightarrow \text{Action} $$

The selected content becomes the context.
The shortcut communicates intent.
Cursivis decides how to help without forcing the user to reconstruct their workflow inside another application.

We were not trying to build another chatbot.

We wanted to build an assistant that appears exactly when it is needed, helps the user move forward, and then disappears back into the workflow.

What it does

Cursivis Next is a context-aware Windows AI assistant that works across the applications people already use.

Its core workflow is intentionally simple:

Select → Trigger → Understand → Respond → Refine or Act

A user highlights text and invokes Cursivis using a global keyboard shortcut. Cursivis captures the selected content, understands its context, and generates the most useful response.

The same interaction adapts to many situations:

An article becomes a concise summary.
A difficult paragraph becomes easier to understand.
A foreign-language passage becomes a translation.
An email becomes clearer or more professional.
A selected word receives a definition.
Code can be explained, reviewed, or debugged.
Notes can become structured action items.
A prompt can be improved before being sent to another AI system.
An image or screen region can be described and analyzed.
Generated text can be inserted back into the original application.

The user does not need to memorize a different command for every situation.

The interaction remains consistent while the intelligence adapts to the context.

Smart Mode

Smart Mode is designed for speed.

Cursivis examines the selected content, infers the most useful transformation, generates the result, and presents it immediately.

Instead of always displaying a generic label such as “Processing,” Smart Mode can identify the task it is performing and communicate that clearly to the user.

Guided Mode

Some selections can support several useful actions.

Guided Mode creates a compact set of options based on the actual context, such as:

Summarize
Explain
Rewrite
Translate
Extract key points
Create a custom task

These options are generated dynamically rather than being the same fixed menu for every selection.

Screen and image understanding

When no text is selected, Cursivis can open a screen-region selector.

The user chooses a meaningful section of the screen, and only that selected region is sent for multimodal analysis.

This allows Cursivis to understand:

Screenshots
Error messages
Images
Diagrams
Interfaces
Visual documents
Text that cannot be selected normally

Result Panel

Responses appear inside a lightweight floating Result Panel instead of forcing the user into another full application.

From the Result Panel, users can:

Refine the result
Request more options
Insert or replace text
Undo supported changes
Continue into Take Action
Close the interaction
Preserve the original context throughout follow-up requests

Results can also be copied automatically, reducing the number of manual steps required.

Live Mode

Live Mode turns Cursivis into a conversational voice assistant while preserving the current desktop context.

A user can select text, start Live Mode, and ask natural follow-up questions such as:

“What does this word mean?”
“Explain this in simpler language.”
“Rewrite this more professionally.”
“What is wrong with this code?”
“Look at this image and tell me what it contains.”
“Guide me through this interface.”

Live Mode includes bounded tools for:

Retrieving the text selected when the session began
Understanding the active application
Copying useful text
Inserting text into the original application
Analyzing an explicitly selected screen region
Starting Navigation Guidance
Continuing into approved browser workflows
Saving and deleting explicit user-controlled memories
Cancelling the current session

The goal is not unrestricted computer control.

The goal is a focused conversational assistant that understands the user’s current work and uses clearly defined tools only when requested.

Smart Dictation

Smart Dictation goes beyond raw speech-to-text.

It records the user’s speech, transcribes it, improves punctuation and structure, and attempts to insert the polished result into the intended application.

When direct insertion is unavailable, Cursivis uses a safe clipboard fallback.

Prompt Optimizer

The Prompt Optimizer helps users improve prompts before sending them to an AI model.

It can transform an incomplete or ambiguous instruction into something clearer, more structured, and easier for an AI system to follow.

Navigation Guidance

Navigation Guidance helps users complete unfamiliar desktop tasks without taking uncontrolled actions on their behalf.

The interaction follows a visible, user-controlled loop:

Observe the permitted application or display.
Explain one clear next step.
Wait for the user to perform it.
Verify that the interface changed.
Continue with the next step.
Stop when the task is complete or cancelled.

The user remains in control throughout the process.

Take Action

Cursivis can move beyond generating text through a structured browser-action workflow.

For supported tasks, it can interpret an instruction, prepare a browser-side action plan, validate it, and continue through a safety-aware execution layer.

Low-risk actions can be handled efficiently, while sensitive or high-impact actions require confirmation.

How we built it

Cursivis Next is a standalone Windows application built using:

C#
.NET 8
WinUI 3
OpenAI Responses API
OpenAI Realtime API
OpenAI audio transcription
Windows UI Automation
Chromium extension APIs
Native Messaging
Inno Setup

We separated the project into clear layers:

Global Keyboard Shortcuts
            ↓
Windows Context Capture
            ↓
Cursivis Application Layer
            ↓
OpenAI Reasoning and Realtime Services
            ↓
Result, Voice, Guidance, or Approved Action

Windows application layer

The WinUI application manages:

Global shortcut registration
Shortcut conflict detection and fallback
Settings and onboarding
Context Orb presentation
Result Panel presentation
Live Mode presentation
Screen-region selection
Smart Dictation
Themes
Startup behavior
Overlay position persistence
Local settings and memory

Context capture

Capturing a selection reliably across Windows applications required more than simply reading the clipboard.

Cursivis uses a layered strategy combining:

Windows UI Automation
Focused-control selection detection
Foreground-window identity checks
A protected clipboard fallback
Clipboard snapshot and restoration
Bounded timeouts
Context lifetime validation

This allows it to work across browsers, desktop editors, documents, and other supported applications without permanently replacing the user’s clipboard contents.

OpenAI integration

Different OpenAI capabilities are used for different parts of the product:

The Responses API handles text and image reasoning.
The Realtime API powers low-latency Live Mode conversations.
Audio transcription powers Smart Dictation.
Structured tool definitions expose only approved Live Mode capabilities.
Model-generated options power Guided Mode and refinement.

Desktop overlays

The floating orb, Result Panel, Live Mode panel, region selector, and guidance surfaces required native Windows integration beyond ordinary XAML.

We used:

Transparent compositor backdrops
Native borderless windows
DPI-aware window regions
Monitor-aware positioning
Draggable and resizable overlay surfaces
Saved placement
Non-activating utility windows
Explicit focus restoration

Browser action layer

For supported browser workflows, Cursivis uses:

A Chromium extension
A Windows native-messaging host
An authenticated local bridge
Structured action planning
Safety classification
Confirmation for sensitive actions

Model output and browser input are treated as untrusted until they pass validation.

Privacy and persistence

Cursivis stores settings, shortcut mappings, overlay positions, approved tasks, and explicit memories locally.

The OpenAI API key is protected using Windows user-level encrypted storage instead of being written into an ordinary text configuration file.

We also isolated Cursivis Next from previous installations by giving it separate:

Storage paths
Installer identity
Startup registration
Native-host identity
Browser bridge identity
Shortcut state

Migration is bounded, whitelisted, copy-only, and non-destructive.

Challenges we ran into

The hardest part of Cursivis was not generating intelligent output.

The hardest part was making every invisible part of the interaction reliable.

Reliable selection capture

Different applications expose selected text in different ways.

A normal text box, Chrome page, browser address bar, Electron application, PDF reader, and custom editor may all behave differently.

During testing, a user selected the word “extempore” inside Chrome, but Windows UI Automation returned the browser’s Google search URL instead.

Solving problems like this required:

Reader precedence rules
Foreground-window validation
Protected clipboard capture
Clipboard restoration
Browser-specific safeguards
Real cross-application testing

Global shortcuts

Global shortcut reliability became one of the deepest engineering challenges.

We encountered:

Shortcuts already owned by Windows or other applications
Persisted shortcuts differing from active shortcuts
Registration rollback failures
Commands disappearing after another operation
Async code continuing on the wrong Windows thread
Multiple installations competing for the same identity

We discovered that native Windows hotkey ownership is tied to the registering window thread.

Cursivis therefore had to perform registration, replacement, rollback, and unregistration through the correct WinUI owner thread.

It now distinguishes between:

The configured shortcut
The active shortcut
A conflicting shortcut
A safe fallback shortcut

If the preferred shortcut is unavailable, Cursivis visibly reports and persists an available alternative rather than silently disabling the feature.

Transparent Windows overlays

A transparent XAML background does not automatically create a truly transparent desktop window.

We encountered:

Opaque swap-chain rectangles
Black edges around rounded windows
Native borders returning after movement
DPI-dependent clipping
Dragging conflicts with buttons
Resize behavior
Position persistence
Focus stealing

Creating an orb that visually feels like part of the desktop required native window regions, compositor handling, DPI-aware geometry, and screenshot-based validation.

Realtime interaction

Live Mode had to coordinate:

Microphone capture
Audio buffering
Realtime events
Interruption
User and assistant transcripts
Selected context
Tool calls
Image analysis
Navigation Guidance
Repeated start and stop
Cancellation
Connection failures

A voice assistant only feels natural when all of these transitions remain coherent.

Balancing action with safety

An AI assistant becomes more useful when it can act, but action also creates risk.

We had to distinguish between:

Informational operations
Reversible changes
Browser-side interactions
Sensitive submissions
High-impact or irreversible actions

Cursivis uses bounded tools, structured validation, explicit permission, confirmation, and undo support wherever possible.

Accomplishments that we're proud of

We are proud that Cursivis became more than a demonstration of an AI response.

It became a complete interaction system connecting selection, shortcuts, reasoning, multimodal input, voice, overlays, insertion, browser workflows, safety, persistence, and recovery.

Some of the accomplishments we are most proud of include:

Building a shortcut-driven AI assistant for Windows.
Capturing selected content without permanently replacing the clipboard.
Supporting both immediate Smart Mode and adaptive Guided Mode.
Falling back to image-region analysis when text is unavailable.
Preserving context across refinement, insertion, replacement, and undo.
Building a Realtime voice mode with selected-text and screen-analysis tools.
Adding explicit user-controlled memory rather than silently saving conversations.
Building Navigation Guidance that observes, waits, verifies, and continues.
Creating a structured browser-action layer with clear safety boundaries.
Making shortcut conflicts visible and recoverable.
Isolating the new product from older installations without deleting user data.
Protecting the API key through Windows encrypted storage.
Building a self-contained Windows installer and startup system.

At one of our latest verified engineering checkpoints, Cursivis passed:

167 unit tests
71 integration tests
A clean x64 Release build with zero warnings and zero errors

Those tests cover behavior users should never have to think about:

Clipboard restoration
Shortcut persistence
Shortcut rollback
Thread ownership
Cancellation
Storage migration
Safety validation
Error recovery
Browser identity
Settings persistence

What we learned

The biggest lesson was that the quality of an AI product is determined by far more than the intelligence of its model.

A powerful model cannot compensate for:

Capturing the wrong selection
Losing a keyboard shortcut
Stealing focus
Damaging clipboard contents
Displaying a broken overlay
Forgetting context
Executing an action the user did not intend

We learned that:

Context capture is often harder than reasoning.
Selection can be a better prompt than a blank text box.
A shortcut can express intent faster than another menu.
Invisible reliability is a real product feature.
Users trust actions when they are visible, bounded, reversible, and cancellable.
Desktop AI must respect focus, DPI, accessibility, clipboard state, operating-system lifecycle, and native thread ownership.
The best AI interaction often asks the user to explain the least.

Most importantly, we learned that AI does not need to dominate the workflow to transform it.

It can appear for a few seconds, understand what matters, help the user move forward, and then disappear again.

What's next for Cursivis-Next

Our next goal is to turn Cursivis Next into a polished and dependable Windows productivity layer that users can install and understand immediately.

We plan to continue improving:

Selection reliability across browsers, editors, documents, and PDFs
Realtime voice stability
Interruption handling
Image and screen understanding
Navigation Guidance
Safe browser workflows
Accessibility
Keyboard-first interaction
Local and transparent memory
Startup reliability
Application-specific context adapters
Evaluation systems for action correctness and safety
Installer and update experience
A future macOS version

We also want to create a developer-facing capability system so new context tools can be added without weakening the safety boundaries of the core application.

The long-term vision remains simple:

Cursivis should make AI feel less like a destination and more like a capability already present inside the work.

The cursor should not merely point at information.

The selection should not merely highlight it.

And the shortcut should not merely open another application.

Together, they should become the beginning of understanding—and, when the user chooses, the beginning of action.

Built With

accessibility
browser-automation
chromium-extension
clipboard
computer-vision
context-aware
csharp
desktop-app
dotnet
hotkeys
image-analysis
inno-setup
multimodal
native-messaging
navigation-guidance
openai
productivity
realtime-api
responses-api
screen-capture
smart-dictation
ui-automation
voice-ai
windows
winui3