FirstRing

1. FirstRing: title card with Ringo balancing the ring on his nose and "Before you answer, FirstRing checks."
2. It picks up before you do: headset Ringo next to a ringing phone, with the Photon + Twilio route.
5. It judges the request, not the caller ID: a warning, a message and an actual extraction attempt, side by side.
3. A real call, verbatim: the Nov 7 live call, word for word, up to "can't help obtain sign-in codes."
8. Fast enough for a phone call: 1.8 s of model time per turn, with the measured timeline and the speed techniques.
12. An offer that can't be revived: the four-step revocation story.
6. Nemotron proposes. Rust decides: the nine tools and the four checks Rust runs before anything happens.
9. Every decision has a receipt: the console's per-call receipts.
13. Share the tactic, never the person: the Firebreak scene, with number reputation.
4. Codes, gift cards and remote access stay with you: the scam waveform stopped at the ring.
11. Ring my phone first: two phones linked through the ring for the private handoff.
10. Answers in your own voice: microphone scene for the ElevenLabs voice cloning.
7. Nemotron, self-hosted on H100: the Brev GPU scene with model and serving facts.
15. One Rust process owns every decision: the system map with the tech stack.
14. What happened, and why: the dashboard screens.

Inspiration

Most phone scams don't start with "give me your code." They start as a normal call — a clinic confirming an appointment, a bank checking a charge — and pivot halfway through. Today's call screeners ask for a name and reason at hello and then hand the call over. We wanted a screener that keeps guarding until the call is actually in Sarah's hands, and that never lets an AI quietly make a decision a person should make.

What it does

FirstRing answers unfamiliar calls to Sarah, a fictional older adult, through a Photon phone number. A disclosed assistant greets the caller (asked directly, it says it is automated) and NVIDIA Nemotron 3.5 Lightning listens turn by turn, returning exactly one native tool call per step from nine tools: record an evidence-quoted assessment, ask a clarifying question, take a message, end the call, look up known tactics, look up a trusted contact, offer Sarah a handoff, start a trusted callback, or publish a tactic. Callers can talk over it at any time, and replies the caller has already talked over are dropped.

Real call, verbatim (Nov 7, 22:10 Central): a caller says they're from Sarah's clinic, asks to change an appointment, then asks for "the 6 digit code." Nemotron first asks whether it's an appointment reference or a sign-in code and tells the caller not to read it aloud. When the caller ties it to a code they sent, the assessment flips to authentication_secret, Rust blocks the call, saves a minimized incident note, and plays a protective goodbye: "I can't help obtain sign-in codes."
Legitimate call, ring my phone first: the caller is held while Sarah's own phone rings on a separate, private leg. She hears a short summary and chooses on the keypad: talk, take a message, or decline. Only her keypress can bridge the call, and the offer expires after 30 seconds.
Pivot to a scam: if the caller asks for a verification code, remote access, a gift card or a payment redirect, the pending offer is revoked immediately. A stale keypress on the old offer is rejected, and FirstRing can start an independent callback to the institution's number from Sarah's trusted directory — never a number the caller supplied — or reach an enrolled family contact. The suspicious caller never joins.
Answers in your own voice: the owner can enroll over a real phone call (consent, then a short recorded passage), preview the result, and switch FirstRing to speak in a cloned voice via ElevenLabs Instant Voice Cloning. A familiar voice changes how FirstRing sounds, never what it is allowed to do — it is never treated as proof of identity, and the clone can be deleted at any time.
Neighborhood Firebreak: scammers reuse scripts, not phone numbers. Each stopped call becomes a minimized, 24-hour tactic capsule (requested action + pretext category — no names, numbers, quotes or transcripts). A second household subscribes to that feed, so the same tactic from a new number is recognized on its first ask, while a benign caller from a new number still gets through. Presented-number reputation is HMAC-keyed, scoped, expiring, correctable and audited, and is context for the model — never a verdict.
Operator console: every step of a call renders as heard → model → proposed → rule → done, with model backend, tokens, finish reason and latency per request, the consent offer and its revocation, phone-leg states, community memory receipts, and dual-channel call recordings you can play back. Settings cover the household profile, preset voice, owner voice flow and the ring-my-phone-first arm.

How we built it

Nemotron proposes. Rust decides. A Rust 2024 server (axum + tokio) runs each call as an actor with a bounded mailbox. Every proposal is checked against the caller's literal words, the current turn and a monotonically increasing policy epoch before anything is spoken, saved, dialed or bridged. Every quote must be a substring of a committed caller turn. Risk is decided structurally (speech act × direction × requested object), not by keywords, so "my bank warned me never to share codes" passes through as a message. Every side effect is reserved in a durable SQLite action ledger before it is submitted, so retries and late status callbacks can't double-dial or regress a call.
Self-hosted Nemotron on an NVIDIA H100 via Brev: nvidia/Nemotron-3.5-Lightning-30B-A3B in NVFP4 (hybrid Mamba-2/attention MoE, ~3B active parameters) served by vLLM from a pinned, digest-locked recipe with native tool calling, prefix caching and async scheduling, reached only through a tunnel. The NVIDIA API Catalog is the hosted fallback and served our first live call. A custom vLLM tool-parser plugin with a compact XGrammar grammar fixed a live failure where the model emitted thousands of formatting spaces after a valid tool call. We ran the GPU inside a $60 lifetime budget with exact quotes, spend caps, watchdogs and read-back of deletion for every run.
Fast enough for a phone call: a forced named tool per phase, phase- and capability-filtered schemas, thinking off, a stop token at the end of the tool call (same-prefix clarification 2,368 → 765 ms), a compact understanding schema, byte-stable prompts for prefix caching, 600 ms endpointing, pre-rendered closings in the call's own voice, and hard bounds (3.5 s per request, 6 s per turn, four steps) so a slow model yields a safe protective closing, never dead air. Eight model requests on the live call took 1.1–2.9 s (median 1.8 s); on 9 exact production contexts, Brev ran 0.7–2.2 s per request vs 2.5–5.9 s hosted.
Telephony and voice: Photon 2.0 number with SIP voice, routed through a Twilio SIP Domain to Rust. Twilio ConversationRelay carries live text turns over one WebSocket with Deepgram speech-to-text (barge-in, partial prompts, DTMF) and ElevenLabs text-to-speech. Gather is a turn-based fallback. Every voice endpoint verifies Twilio signatures, and the WebSocket additionally requires a per-session token.
Angular 22 + Tailwind 4 console over authenticated server-sent events with Zod-validated state, Last-Event-ID resume and memory-only operator tokens, deployed as a static site on AWS (S3, CloudFront, Route 53, ACM, Terraform) at firstring.life, with the phone server behind a Cloudflare tunnel.
Testing: 370 Rust tests (property-based tests for revocation and terminal-status invariants, official Twilio signature vectors, real WebSocket upgrades, callback races, clone-enrollment and reputation suites) with zero Clippy warnings, plus 57 Angular unit tests and 47 Playwright browser tests. We also built a robot caller: ElevenLabs-synthesized caller clips played through a second Photon line to place real calls to FirstRing, graded against a rubric written before synthesis, with receipts for every run. Model behavior is evaluated separately on production-context matrices and a frozen held-out set.

Challenges we ran into

Consent races are hard: a caller can change their request while Sarah's private offer is still ringing, and a keypress can arrive after the offer is obsolete. We modeled offers with epochs and atomic acceptance so an old "1" can never bridge a revoked call. Phone providers also report call status out of order and sometimes ambiguously, so every outbound leg is reserved once and reconciled rather than retried blindly. Making a 30B-class model decide inside a phone call's latency budget took the forced-tool phase machine, the stop token and the compact grammar. And a six-digit code is ambiguous on its own, so the model has to ask what kind of code it is before it can judge.

Accomplishments that we're proud of

An AI that is useful on the phone without being trusted with the final decision.
Real end-to-end calls: Photon → Twilio → Rust → Nemotron on a self-hosted H100, including a live protective refusal and an automated robot-caller campaign over real telephony.
Privacy by design: the shared tactic memory carries categories and expiry only, and the private summary is minimized before Sarah hears it.
A console that shows receipts, not assumptions: proposals stay visibly separate from executed actions.
Running a GPU on a $60 hackathon budget with guards on every run.

What we learned

Scam detection is the easy half. The product is the safeguard around it: who is allowed to decide, which numbers may be dialed, and what happens when the situation changes mid-call.