Refract

Inspiration

The idea came from a moment every cross-functional team has lived through: an engineer drops a message like "we're gating the pricing engine behind a canary release," and everyone in the thread - Sales, Marketing, Design - just nods. Nobody stops to ask what it means, because stopping feels slow, or it exposes a gap in what you know in front of your whole team. So the confusion doesn't get resolved in the moment. It resurfaces three weeks later as a missed deadline, a promise a salesperson shouldn't have made, or a launch narrative that quietly contradicts what engineering actually shipped.

We'd all either caused this problem or been on the receiving end of it. The tools that exist - Slack AI, ChatGPT in a tab, internal wikis - all require you to notice the gap, open something, and ask. That friction is exactly why the gap never closes in real time. We wanted something that lived inside the conversation and stayed completely silent until the exact moment someone needed it.

The second idea, which became the actual core of the product, came a bit later: the problem isn't just "translate jargon." It's that the same sentence should mean something different depending on who's reading it. A Salesperson reading "canary release" needs to know if their renewal date is at risk. An Engineer reading "pipeline velocity" needs it explained through kanban and throughput, not a dumbed-down glossary entry. That's not one translation - it's a refraction: one input, split into role-specific framings, the way a prism splits light. That's where the name came from.

What it does

Refract sits inside a live, multi-user team chat. Highlight any phrase in any message, and a small Ask AI button appears. Click it, and you get:

A plain-language explanation reframed in your own discipline's mental models - not simplified, reframed.
2–3 impact bullets describing what this actually means for your work, hedged as inferences and grounded in your team's shared context brief.
Verified citations - every bullet that draws on a specific fact shows exactly which sentence in the brief or the source message it came from, checked server-side so the citation can't be a fabrication.

The same highlighted text, viewed by five different roles, produces five genuinely different explanations - you can watch this happen live by toggling a role switcher inside the popover itself. You can ask a natural follow-up without leaving the popup, and share a good explanation back to the whole thread with one click. And critically, the widget isn't wired to chat specifically - it works identically on a static spec document, because the whole mechanism is just a DOM contract (data-message-text + data-message-role) that any surface can expose.

How we built it

We split the build into two tracks that met at a clean interface: a chat host (FastAPI + SQLite + WebSocket, handling real-time multi-user messaging, a seeded demo conversation, and an editable team context brief) and the brain (the /explain pipeline - prompt design, the Mistral integration, and the highlight widget).

The /explain pipeline ended up as a genuine chain, not one prompt:

A per-IP rate limiter and an in-memory cache guard the front door.
A cheap ministral-3b call acts as a gate - deciding whether a highlight is substantive enough to deserve the expensive call, so "ok" and "thanks!" don't burn a full completion.
The main mistral-small call does the actual reframing, with the system prompt (kept in an editable .md file, not hardcoded in Python) enforcing strict JSON output, hedged language, and role-aware reframing in both directions - tech → non-tech and non-tech → tech, treated as equally important.
The response is parsed defensively, re-validated against a Pydantic schema, and every citation is checked to be a verbatim substring of the brief or the source message before it's shown to the user. Fabricated citations get dropped silently rather than trusted.
If Mistral is unreachable, a small table of pre-verified "known good" responses covers the scripted demo path so a live outage doesn't sink a presentation.

On the frontend, the highlight widget listens for text selection anywhere inside an element carrying the right data attributes, floats an "Ask AI" button near the selection, and renders the result in a popover that repositions itself to stay on-screen. The role toggle re-fires the same /explain call with a different reader_role and the content morphs in place - which is what makes the "same text, different role" moment demonstrable in a single browser window instead of needing two.

We ran a 10-scenario × 5-role evaluation harness against the real pipeline - 50 live Mistral calls - to catch prompt regressions and confirm quality before every significant prompt change, rather than eyeballing single examples.

Challenges we ran into

Latency vs. quality trade-off. We benchmarked mistral-small against mistral-medium on identical prompts before locking a model. Medium's answers were marginally richer, but latency jumped from about 3 seconds to over 12 - unacceptable for something rendering in a hover popover. We chose to invest further effort in prompt design and grounding instead of reaching for a bigger model, which turned out to be the higher-leverage lever anyway.

Reliability without an LLM's cooperation. LLMs don't reliably emit clean JSON, and they don't reliably tell the truth about their own reasoning. We couldn't assume either. The defense ended up layered: strict JSON-mode requests, a Pydantic re-validation pass, a plain-text fallback if parsing fails, and - the part we're proudest of - actually verifying every citation against the source text rather than trusting the model's claim that it was grounded. If the model invents a quote, we catch it and drop it before the user ever sees it.

The gate model was too aggressive. After wiring the ministral-3b pre-filter to save cost on trivial highlights, we found it sometimes misclassified real jargon like "canary release" as trivial chit-chat - a false negative that would have quietly broken the demo. It was a good reminder that even a "simple" classification step needs its own evaluation, not just an assumption that a smaller model will behave sensibly on the edge cases.

24-hour scope discipline. With the core "highlight → role-aware explanation" loop working by the midpoint, it was tempting to keep adding features. We forced ourselves to treat the same-text-different-role moment as the one thing that couldn't be cut, and sequenced everything else follow-up Q&A, share-to-thread, the second demo surface - as things to add only once that core was rock solid.

Accomplishments we're proud of

A model-chained pipeline that makes a real cost/latency decision per step, not a single prompt dressed up in a UI.
A citation system that verifies its own grounding claims instead of just asking the model to "please be accurate."
Proving generality live: the exact same widget component works unmodified on a second, completely different surface (a static document) because the contract between the widget and its host is just two DOM attributes.
Zero unhandled failure paths in the explain pipeline - missing API key, malformed JSON, provider timeout, and rate limiting all degrade to a defined, user-friendly state instead of a crash.

What we learned

The most valuable lesson was that prompt quality and system design are inseparable from reliability engineering. A clever prompt is worth very little if a single malformed response can crash the UI, and a "citations" feature is actively dangerous if you don't verify what the model claims to be citing. We also learned, very concretely, that bigger models aren't automatically better for a product - the right model is the one that fits the actual latency budget of the interaction, and the quality gap can often be closed with better grounding and prompt structure instead of more parameters.

What's next

The architecture is deliberately provider-agnostic and surface-agnostic. The next real steps would be wiring the same widget into an actual team tool (Slack, Notion) rather than our purpose-built demo chat, and extending the "team context brief" into something that can pull from linked documents automatically instead of being manually maintained.