Relay

Inspiration

What it does# Relay

A translator that learns from its mistakes — so a wrong translation in a crisis only ever happens once.

Inspiration

Generic translation tools are trained on the languages with the most data on the internet — which means the languages spoken in the places where translation errors are most dangerous (disaster zones, refugee intake lines, a field clinic with no working WiFi) are exactly the ones those tools handle worst. Haitian Creole, Tigrinya, Quechua, Wolof: sparse parallel corpora, sparse QA, and a translation API that will confidently hand a field worker the wrong phrase for "he is allergic to penicillin" with no indication it might be wrong.

The failure mode that stuck with us wasn't bad translation — it was silent bad translation. A tool that's wrong and knows it's wrong is useful. A tool that's wrong and sounds certain is a liability. That gap — between confidence and correctness — became the actual product to build.

What it does

Relay is a two-way conversational translator built for high-stakes, low-resource-language situations: medical triage, shelter intake, asylum interviews. It does two things generic translators don't:

1. It shows its work. Every translation comes back with a confidence signal (High / Medium / Low, shown as signal-strength bars, not just a color) built from retrieval similarity and self-consistency across sampled completions — not vibes.
2. It remembers corrections. When a field worker fixes a mistranslation, that correction is embedded and stored. The next time someone asks something similar, Relay retrieves the verified correction instead of trusting the model again. Say "I need water" wrong once, correct it once, and every future phrasing close to it comes back right — instantly, without another model call.

That loop — translate → flag uncertainty → correct → remember — is the entire point of the project.

How we built it

• Backend (FastAPI, Python): a translation orchestrator that embeds the input, checks a Chroma vector store of verified corrections first, and only falls through to an LLM call if nothing matches above a tuned similarity threshold. The LLM path pulls in RAG-retrieved domain glossary terms and samples two completions to score self-consistency.
• Correction memory & glossaries: two separate Chroma collections — one for user-verified corrections, one for seeded domain terminology (medical triage, shelter intake, asylum interview, general) — each filtered by language pair and domain before retrieval.
• Provider abstraction: LLM, STT, and embedding calls all sit behind an interface, selected by environment variable. Embeddings run locally (all-MiniLM-L6-v2) so the core retrieval loop has zero external dependency. If no API key is configured, or a call fails, everything degrades to a small bundled rule-based translator — the app never shows a broken state, which matters a lot when your "production environment" is a hackathon stage with conference WiFi.
• Frontend (Next.js, TypeScript, Tailwind): a two-party conversation view with mic input (MediaRecorder, falling back cleanly to text-only when unsupported), inline edit-in-place corrections, and browser-native speech synthesis for zero-backend-dependency text-to-speech. All API responses are runtime-validated with zod at the network boundary rather than trusted blindly.
• Confidence scoring: a small, pure, independently unit-tested function — max(retrieval_similarity, self_consistency) bucketed into High/Medium/Low — deliberately boring and explainable rather than another opaque model call.

Challenges we ran into

• Tuning the correction-memory threshold. Too low, and unrelated phrases start getting served the wrong stored correction. Too high, and near-exact rephrasings miss the correction they should hit. We landed on 0.86 cosine similarity as the point where recall and precision felt right for the domain — documented as a single named constant rather than a magic number scattered through the codebase.
• Async correctness. The UI lets someone correct a translation while, in theory, another request for the same phrase is in flight. We had to explicitly guard against a correction being submitted for a translation that hadn't actually resolved yet, rather than assuming a fire-and-forget write had landed.
• Making "offline mode" actually mean something. It's easy to build a mock fallback that only gets exercised in a demo script. We built it as the default path — no API key needed at all to run the full loop — so mock mode isn't a special case, it's just what happens when nothing else is configured.
• Build-time fragility. We initially pulled in a Google Fonts family at build time, then realized that's exactly the kind of network dependency the project is supposed to protect against — a flaky venue connection shouldn't be able to break the build. Pulled it out in favor of system font stacks.
• Signaling confidence without a traffic light. Red/yellow/green is the obvious choice and the wrong one for accessibility and for anyone colorblind reading a screen in a hurry. We ended up encoding confidence as signal-strength bars (a shape) plus a text label, with a single accent color doing double duty rather than three separate meaning-bearing colors.

Accomplishments we're proud of

• The correction loop genuinely works end to end, not just on the happy path: type a phrase, get it wrong, correct it, ask again in different words, watch it come back right.
• The app runs with literally zero configuration — no API keys, no external services beyond a local embedding model — which was a deliberate insurance policy against demo-day failure, not an afterthought.
• Confidence scoring is deterministic and testable, so "why did it say Low confidence here" has a real answer instead of "the model felt uncertain."

What we learned

That the hard part of building a trustworthy translation tool isn't the translation — it's the epistemics around it. Knowing when to doubt your own output and what to do with a correction once you have one turned out to be a much more interesting engineering problem than picking a bigger model. We also relearned the value of graceful degradation as a first-class design constraint rather than error handling bolted on at the end — building the mock path first, before the real providers, meant every later feature had a working fallback from day one instead of us retrofitting one under deadline pressure.

What's next for Relay

• Running scripts/generate_glossary.py against genuinely low-resource languages at scale, with manual review, rather than the hand-seeded starter glossaries.
• Packaging the whole thing behind docker-compose up for true two-command setup.
• Expanding self-consistency checking beyond token-level Jaccard similarity toward something that understands paraphrase, not just word overlap.

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for Relay

Built With

• anthropic
• chromadb
• claude
• embeddings
• fastapi
• httpx
• javascript
• llm
• machine-translation
• mediarecorder-api
• nextjs
• openai
• pydantic
• pytest
• python
• rag
• react
• sentence-transformers
• tailwindcss
• typescript
• uvicorn
• vector-search
• web-speech-api
• whisper
• zod