Ask the Market

Inspiration

Every Lloyd's of London syndicate publishes its results in a 50 to 200 page annual-report PDF. The people who allocate capital across syndicates, members' agents, reinsurers and capital providers, read those PDFs by hand every results season. A question as basic as "which syndicates grew premium while improving their combined ratio last year" can take an analyst days, because the answer is scattered across a hundred separate documents in a hundred slightly different layouts. We wanted to ask the whole market a question in plain English and get the answer in seconds, with every number traceable back to the page it came from. In insurance, an answer you cannot check is worthless, so provenance had to be the product, not a footnote.

What it does

Ask the Market is a research desk for the Lloyd's syndicate market. You type a question in plain English and get a ranked, cited answer. Three surfaces sit behind one question box: a research desk over the 25 syndicates we reconciled figure by figure against their source PDFs, where you click any number to open the actual report page with the figure highlighted; a syndicate universe covering all 132 syndicates we extracted, 2020 to 2025, searchable and sortable with a profile page each; and a market-in-charts view of Lloyd's whole-market aggregates from 2016 to 2024, every series an interactive chart. Ask "rank syndicates by combined ratio in 2024" and you get the full field of around ninety. Ask "how has Chaucer's combined ratio trended since 2020" and it resolves the name to its number, pulls the series, and writes a cited summary. Ask something it cannot source and it tells you, instead of inventing a number.

How we built it

One Amazon Aurora PostgreSQL database (Serverless v2) does two jobs at once: relational facts for the rankings and pgvector for the report narrative, joined in a single SQL query. Each figure is stored in its native filing currency, with a GBP-normalised value for fair comparison and a page number for provenance. The language model never writes SQL. A question goes to Claude Sonnet 4.6 on Amazon Bedrock, which picks one of eight allowlisted query intents and fills its parameters. A zod validator checks every parameter. Only then does a fixed, parameterized SQL template run, on a read-only Aurora role, inside a READ ONLY transaction with a statement timeout and row caps. The answer is composed only from the rows that came back, and every figure is cited to its provenance column. A number cannot be hallucinated because the model never gets to write the number. The frontend is Next.js 15 and React 19 on Vercel with hand-built interactive SVG charts; Titan Text Embeddings v2 produce the 1024-dim vectors; an offline pipeline extracts figures from the PDFs, reconciles the cited set against the source pages, and renders those pages to highlighted PNGs.

Challenges we ran into

Trust at the figure level: it is easy to get a model to produce a confident table, hard to guarantee every number is real, so we solved it structurally (the model routes and composes, it never does arithmetic). Two scopes, one product: the 25 page-cited syndicates are the depth layer, the 132 extracted are the breadth layer, and the market aggregates are a coarser source going back to 2016; we split them into three surfaces and made scope an execution setting, so the Ask box opens to all 132 while the cited desk stays at 25. Double-counting in the source data: reports list subtotal rows and duplicate line spellings, so we taught the parser to prefer the authoritative total row and verified zero leakage. Cross-currency comparison: syndicates file in GBP, USD and EUR, so we store native values for citation and a GBP value for ranking.

Accomplishments that we're proud of

Every figure on the cited desk is reconciled against its source page, and you can click to prove it. The safety model is real, not a prompt: the LLM cannot reach the database except through eight allowlisted, parameter-checked, read-only intents. One database does relational and vector work in a single query, exactly what the AWS-database brief asked for. And we went from 25 syndicates to all 132 without losing the citation guarantee.

What we learned

In finance, provenance is the feature; the click-to-source moment does more for trust than any amount of polish. Constraining the model beats prompting it: an allowlist plus a validator plus a read-only role give you a guarantee a system prompt never can. And granularity is a product decision: per-syndicate data and market aggregates have different reach and different sources, and naming the three layers fixed the confusion.

What's next for Ask the Market

Expand the page-cited set from 25 toward the full 132 so every figure is click-to-source; add FX coverage for 2020, 2021, 2024 and 2025 so cross-currency premium tables span every year; go beyond combined ratio and premium into reserves, capital and reinsurance structure; and add alerting so a capital allocator hears the moment a syndicate's combined ratio crosses a threshold, citation attached.

Built With

• amazon-aurora
• amazon-bedrock
• amazon-titan-embeddings
• anthropic-claude
• aurora-serverless-v2
• next.js
• node-postgres
• node.js
• pdf.js
• pgvector
• postgresql
• react
• tailwindcss
• typescript
• vercel
• vercel-ai-sdk
• zod