Idun Compass

The blind spot

Enterprises spent the last two years buying AI seats. Claude, Copilot, Gemini, ChatGPT, plus whatever their own teams built on the side. Then renewal comes up, and nobody in the room can answer three questions: what are people actually doing with it, what is each of those things costing, and what should we do more of.

The dashboards that ship with those products count seats and messages. A seat count is not a use case. "4,200 messages last month" tells a CIO nothing about whether to renew, industrialise, or cut.

We run this audit for a living. We have done it by hand in organisations from 39 seats to 15,000 users, and the manual version takes twenty consulting days. What surprised us is that most of those days are not judgement. They are reading — reading redacted prompts, grouping them, counting them, pricing them. That is agent work.

Idun Compass is that reading, done by a fleet of agents.

What it does

Point Compass at an organisation's AI usage exports — Claude Enterprise, Microsoft 365 Copilot, ChatGPT Enterprise, Gemini Enterprise, plus OTLP traces from internally built agents — and it returns three things.

The map. A catalogue of the use cases that actually exist, discovered from data rather than from a workshop. Each entry says what the use case is, which teams run it, how often, whether it survived past week two, and whether three other teams quietly built the same thing.

The count. Cost per use case — token volume and licence seats against real usage — and time saved, converted at an hourly rate built from that company's own roles. Every figure is stamped measured or projected. The two never blur.

The call. What to industrialise first, what to cut, who to train on what. Ranked, and every recommendation carries a trail back to the events that justify it.

How we built it

Seven agents on the Agent Development Kit, running on Cloud Run, fanned out over Pub/Sub, checkpointed in Firestore so a run over a year of logs can resume instead of restarting.

Intake normalises five incompatible export formats into one UsageEvent schema. Gemini 3.5 Flash absorbs the schema drift a parser cannot.
The redaction gate is the one deliberately non-agentic step. Sensitive Data Protection plus deterministic rules strip names, secrets, customer identifiers and free-text PII before any model sees a byte, and emit a signed redaction receipt per batch.
Discovery embeds the redacted prompts, clusters them, and has Gemini 3.5 Pro name each cluster as a use case with a canonical definition. This is the step that turns 40,000 sessions into sixty named things.
Signals reads what the labels miss: abandonment — someone tried this for nine days and stopped — retry storms, and near-duplicate assistants across teams.
Economics joins each use case to tokens and seats, prices it, estimates time saved, and refuses to emit a number it cannot source.
Interview is the agent that talks to humans. The data cannot say why someone stopped. This one writes a short targeted script per open question, runs it interactively, asks clarifying follow-ups, and folds the answers back into the catalogue as first-class evidence.
Arbitration reads the whole catalogue and writes the call.

Google Cloud: Cloud Run for services and jobs, Firestore for the catalogue and resumable run state, Pub/Sub for fan-out per tenant and period, Cloud Storage, Sensitive Data Protection, Secret Manager.

Models: Gemini 3.5 Pro for naming, arbitration and the interview; Gemini 3.5 Flash for high-volume normalisation and classification; Gemini embeddings for clustering; and Gemma 3, self-hosted, for the sovereign path — several of the organisations we sell to will not let prompt content leave their perimeter at all, and open weights they can run themselves is the only honest answer to that.

What we learned

The headline use case is never the finding. Everyone expects the answer to be "they use it for email". The finding sits in the long tail: the same use case rebuilt independently by six teams, each with a private assistant, none of them shared. That duplication is the cheapest win in the whole report, and it is invisible to every seat-count dashboard on the market.

Redaction costs less signal than we feared. We assumed stripping content would gut clustering. It did not. The shape of a session — length, tool calls, retries, abandonment — carries most of the use-case signal, and the redacted text carries the rest.

Interviews and data are not interchangeable, and agents make that cheap. Our practice has one rule we refuse to break: never data instead of interviews. We assumed that made the work un-automatable. It does the opposite — it makes the interview targeted. The agent only asks about what the data flagged, so a 45-minute discovery call becomes four questions.

Challenges

Ordering as a safety property. "Redact before the model" is easy to say and easy to break with one refactor. We made it structural instead: the raw bucket is not readable by the service account the analysis agents run under, so bad wiring fails closed rather than leaking.

Five exports, no common schema. Every vendor disagrees about what an event even is. One counts conversations, one counts messages, one counts active days.

Honest numbers. The measured-or-projected stamp sounds like a formality. It is the hardest constraint in the system, because it means an agent has to know when it is estimating and say so, out loud, in the deliverable.

Cost. Naming sixty clusters is cheap. Classifying forty thousand sessions is not. Flash carries the volume, Pro carries the judgement.

What's next

Connectors instead of exports, a resident mode that runs weekly rather than once, and pushing the whole sovereign path onto self-hosted Gemma so the analysis can run inside an air-gapped perimeter.