Vigile

Inspiration

I worked on payments at Accenture, and on-call was rough. Phone rings at 2am, payments are throwing errors, and I'm digging through logs trying to figure out one thing: have we seen this before, and how did we fix it?

The fix was almost never the hard part. The hard part was finding out whether someone on the team had already solved this six months ago. Most of that knowledge lived in people's heads, or in a Confluence doc nobody updated. So I'd just wait. Or wake people up across time zones. And while I was doing that, customers were abandoning carts.

US e-commerce loses something like $80 billion a year to failed payments, and most teams will tell you they can't really say why a payment failed. It's not a fraud problem. It's a diagnosis problem.

I built Vigile because I wanted the tool I never had at 2am. Something that could look at a wave of failures, group them by what they actually mean (not just keyword match), and tell me whether the team had already seen this before. So I wouldn't spend six hours rediscovering what someone else already figured out.

What it does

Vigile watches your payment logs and helps the on-call engineer figure out what's going wrong, fast.

When payments start failing, it groups them by meaning using pgvector similarity. Not keyword rules. Most payment log lines have dynamic stuff in them (amounts, order IDs, timestamps), so keyword grouping breaks. Embeddings handle that.

It ignores the routine noise. Every e-commerce store has a steady stream of expired cards, wrong CVCs, insufficient funds. Those aren't incidents. Those are just Tuesday. Vigile only creates an incident when a real cluster forms, something unusual, something correlated.

When that happens, it:

• Creates a numbered incident with the dominant issuing bank, US state, processor, transaction count, and how much revenue is at risk
• Pings the on-call engineer in Slack right away with the severity and a link
• Streams a root cause analysis from Claude when you click Analyze, using the team's own past incidents, runbooks, and similar failing logs as context

The part I like most: when Claude explains the incident, it cites a real past incident the team resolved before. The team's own history becomes the playbook.

Triage that used to take 45 minutes now takes about a minute.

How we built it

The big decision was the database. I needed semantic clustering, which means I needed a vector index. Aurora DSQL doesn't have one. DynamoDB doesn't have one. Aurora PostgreSQL with pgvector does, and on top of that, it has foreign keys, JSONB, full-text search, all the stuff you actually want in a real production system. So I put everything in one Aurora cluster: services, logs, incidents, runbooks, and the 1,536-dim embeddings. No separate vector database. One query language. One set of indexes.

The SDK is a small TypeScript package. It batches up to 50 events, retries on failure, and strips out full card numbers before they ever leave the server. Only the BIN (first 6 digits) is sent. Payments teams won't touch anything that could leak a PAN, so I built that in from day one.

Ingestion runs as a Vercel Edge function. It takes the batch, validates it, generates embeddings in one OpenAI call, then bulk-inserts with Postgres unnest(). Single round trip, no row-by-row inserts.

Detection is the part I'm most happy with. It's one pgvector self-join over recent error logs (cosine similarity > 0.85), running every 60 seconds. The same query clusters the failures and computes the dominant issuer, state, and processor, all inside Aurora. No external API calls. The incident's embedding is just the centroid of its member logs.

RCA runs three pgvector queries in parallel (similar logs, matching runbooks, resolved past incidents), feeds the context into Claude through the Vercel AI SDK, and streams the response back to the dashboard token by token.

Slack alerts go out via webhook the moment an incident is created.

The stack: Next.js 14 (App Router), TypeScript, Drizzle ORM, Aurora PostgreSQL 17 with pgvector 0.8, IVFFlat indexing with cosine distance, OpenAI's text-embedding-3-small, Anthropic Claude, and Vercel for everything else.

There's a live /architecture page that queries Aurora directly so you can see the actual pgvector version, vector counts, IVFFlat index size, and measured similarity latency. And a /simulate route that triggers a real Black Friday-style failure burst. That's what powers the live demo.

A couple of choices worth explaining:

I stream Claude directly through the Vercel AI SDK instead of routing through Bedrock plus the AI Gateway. The gateway is great if you need multi-model routing, but I wanted fine control over the streaming token loop because the live RCA is the core experience of the product. The inference layer is decoupled though, so switching to Bedrock-hosted models later is a config change, not a rewrite.

I also denormalized the payment-specific fields (processor, error code, BIN, issuer, state, amount) out of JSONB into top-level columns. The clustering query computes dominant dimensions with MODE() WITHIN GROUP, and that's a lot faster over indexed columns than over JSONB extraction.

A lot of the design thinking came from Aditya Samant's pgvector posts on the AWS Database Blog and the May 2025 benchmark using 10M e-commerce product embeddings. Those were really useful in justifying the single-store architecture.

Challenges we ran into

pgvector recall on log paraphrases. This one took me a while. Two log lines describing the same Stripe timeout would score around 0.5 cosine similarity, way below my 0.85 clustering threshold. They were the same failure but worded differently. I fixed it by templating the embedding input: pull the structured fields (error code, processor, decline reason) and embed a canonical signature instead of the raw message. Scores jumped to about 0.94 on real paraphrases. That single change is what made clustering actually work.

Cold starts on Aurora Serverless v2. I set min capacity to 0 ACUs to keep cost near zero, but then the first query after idle takes 15 seconds. For the demo I bumped it to 0.5 ACU minimum, which costs maybe $7 for the judging window. Worth it.

Serverless connection limits. Vercel functions opening fresh Aurora connections on every invocation hit Aurora's limit fast under load. I had to tune connection handling and verify EXPLAIN latency stayed under 100ms.

Tuning the noise threshold. The first version paged the on-call for every minor decline cluster. Real payment data has so much routine variance that without a meaningful cluster-size threshold, the alerts are useless. Now baseline declines stay in the metrics, not the alerts.

Accomplishments that we're proud of

It's a real working product on a public URL, not a recorded demo. The architecture page queries Aurora live. The simulate button fires actual failures. Claude does its thing with real pgvector retrieval.

The clustering query does a lot of work in very few lines. One self-join computes cluster membership, dominant dimensions, revenue at risk, and severity in one round trip inside Aurora.

PCI-safe SDK from day one. Full card numbers get regex-stripped before send. Only the BIN ever leaves the customer's server. I knew this had to be true from the start because no payments team would adopt anything that could leak a PAN.

The first time I watched Claude pull up a real past incident from the team's history via pgvector similarity and reference it in a new RCA, that was the moment I knew the architecture was right.

What we learned

You probably don't need a separate vector database. For almost everything below billion-vector scale, pgvector inside Postgres or Aurora is the right answer. You get all the relational stuff for free, and adding a separate vector store just creates a second source of truth you have to keep in sync.

How you template the embedding input matters more than which embedding model you pick. Out of the box, the model could not cluster paraphrased log lines. Templating around structured fields fixed it. I spent more time on input formatting than on any model comparison.

Streaming changes how a product feels. The same RCA delivered as a batched response feels like waiting on an API. Token-by-token streaming feels like the system is thinking out loud. You start reading and acting before it finishes.

Foreign keys still matter. Every "modern" architecture I considered would have pushed integrity into application code. Aurora lets the database enforce it. Less code to write, fewer bugs to chase.

The last thing, and maybe the least technical: knowing what matters in payments is more than half the product. The dashboard's credibility comes from showing the right dimensions (dominant issuer, dominant state, processor, revenue at risk), not from the math underneath.

What's next for Vigile

• Stripe and Braintree webhook ingestion so customers don't need the SDK to get started.
• PagerDuty and ServiceNow integration so incidents create tickets in whatever the team already uses, not just Slack.
• A chargeback evidence pack feature using the same pgvector retrieval, pulling similar successful transactions for the same customer and drafting a dispute response with Claude. A friend of mine who runs a watch store is fighting a $16k chargeback right now and spends two hours a month on dispute paperwork. The architecture for this already exists in Vigile.
• Cold-storage tier with Amazon S3 for log retention. Keep the Aurora hot tier bounded to the last 90 days so similarity search stays fast as data grows.
• OpenTelemetry ingestion alongside the SDK for teams already on existing observability stacks.
• Multi-tenant auth and self-serve onboarding.
• Bedrock-hosted embedding models for customers who need all inference within AWS.

Over the next few weeks, the focus is talking to mid-market e-commerce engineering teams, companies between $20M and $300M GMV running on Stripe and Braintree, where every failed transaction shows up directly on the P&L (Profit and Loss document where firm refers to check the actual revenue).

Built With

• amazon-aurora
• anthropic-claude
• next.js
• node.js
• openai
• pgvector
• postgresql
• react
• shadcn-ui
• slack-api
• tailwindcss
• typescript
• vercel
• vercel-ai-sdk
• vercel-cron
• vercel-edge-functions