Last Line

Inspiration

Everyone pastes stuff into ChatGPT they probably shouldn't, API keys, passwords, internal code, and everyone trusts what it says back a little more than they should. Those are the same failure mode from two directions: no verification layer on either side of the conversation. We wanted something that sits in the middle of that conversation and actually checks both directions instead of just hoping for the best.

What it does

Last Line is a Chrome extension that runs directly on chatgpt.com. Before you send a message, it scans your draft for secrets (API keys, passwords, credit card numbers) and prompt injection attempts, and gives you a chance to redact before anything goes out. After ChatGPT responds, it extracts the factual claims from the answer, verifies each one against live web search results (not the model's own training data, so it can't just vouch for itself), and surfaces a 0-100 trust score along with per-claim verdicts. It also flags risky code ChatGPT hands you: hardcoded secrets, SQL injection via string concatenation, eval usage.

How we built it

The extension is a Manifest V3 content script plus service worker, injecting a composer overlay and a floating results panel straight into the ChatGPT DOM. The actual firewall logic (PII/secret detection, claim extraction, verification, hallucination scoring) is a set of pure TypeScript functions shared between the extension and a Next.js backend, so it's unit-testable without a browser and swappable without duplicating logic. Outbound scanning is regex and entropy based, checking for known API key prefixes plus a high-entropy fallback, and Luhn-checking anything that looks like a card number. Inbound verification runs through Groq for the LLM calls (claim extraction, injection classification, judge model) and Tavily for search, streamed back to the page over SSE so the trust panel finishes analyzing while the response is still visibly streaming in.

Challenges we ran into

Getting reliable claim verification without letting the model grade its own homework was the hard part, an LLM checking its own claims against its own training data proves nothing, so every claim has to round-trip through actual web search before a verdict comes back. That's also a latency problem: one message can trigger 6-8 LLM calls end to end (injection check, claim extraction, one verify call per claim, the chat response itself), so we picked Groq specifically because anything slower made the demo feel broken. On the extension side, chatgpt.com's DOM is end to end (injection check, claim extraction, one verify call per claim, the chat response itself), so we picked Groq specifically because anything slower made the demo feel broken. On the extension side, chatgpt.com's DOM is not built for anyone to hook into it reliably, so watching for new messages and finding the composer needed real DOM observation logic and simulated jsdom tests against a fake version of the page to keep it from silently breaking.

Accomplishments that we're proud of

The whole pipeline, outbound scan, redact, send, stream response, extract claims, verify against search, score, and render, runs inline on the actual page with no popup and no separate tab, and it keeps up with a streaming response instead of lagging behind it. The firewall logic itself is fully decoupled from both the extension and the web app, so it's covered by a real unit test suite instead of being tangled into UI code.

What we learned

Verification only means something if it's grounded outside the thing being verified, self-checking LLM output with the same LLM is close to useless. We also learned a lot about how much latency budget matters for anything meant to feel invisible: a firewall nobody notices only works if it's fast enough to disappear.

What's next for Last Line

Broader coverage beyond chatgpt.com (Claude, Gemini, other chat UIs), a configurable sensitivity/allowlist so it doesn't flag things you've already decided are fine to share, and local-only scanning modes for people who don't want even the redaction step touching a network call.