Said Out Loud

Inspiration

Michael Argenyi told the dean of his medical school, in writing, that without accommodations he "was unable to follow lectures, participate in labs, or communicate with patients". He borrowed about $53,000 in his first year for interpreting and real-time transcription, and more than $100,000 across his training. A jury found for him in 2013, and the court entered a permanent injunction against Creighton University plus $478,372.42 in fees and costs.

That is what it costs one person when the system works. Most of the time it is never reached: NCES found only 37% of students who had a disability while enrolled ever told their college. And the part of teaching that decides a grade is often the part nobody writes down, which means listening and writing at once. A student reading captions stops receiving the moment they look down. An apprentice has both hands on a lathe.

What it does

Inside a declared teaching session, Said Out Loud keeps five kinds of sentence and discards everything else: a deadline that moved, a room change, "this will be on the exam", "email me by Friday", and the instructor's answer to a question another student asked. Each becomes a dated ledger row with a verbatim quote, and nothing becomes a todo until the student says yes.

How we built it

The model never returns prose. Not a summary, not a label it made up, not a word a student reads. That is the design, and the rest of the build is downstream of it.

Deciding whether a sentence is an announcement is the entire product, and no pattern separates "the deadline moved to Friday" from "today we will cover recursion", so Amazon Bedrock makes that call. It makes it by returning four things: a category from a closed list, a rationale code from a closed list of ten, a confidence number, and two character offsets into the utterance it was shown. composition-guard.ts then walks the whole parsed response and rejects any string that is not a member of one of those vocabularies, including strings in fields the product never reads, so a model that smuggles a summary into an extra key fails the entire verdict rather than having it quietly ignored. A test hands the guard a perfectly good summary and asserts the verdict is thrown away.

The quote a student sees is sliced out of the utterance locally, by character offset. There is no code path by which a string the model wrote becomes a string the student reads. Asking for offsets is also where this nearly came apart: both Haiku 4.5 and Sonnet 4.5 landed a few characters off, mid-word, on spans like "rt: I am not holding office hours". Models count characters badly. The span is now snapped outward to the sentences it touches, never inward, so a bad offset costs a slightly longer quote rather than a mangled one, with a fuzz test asserting every accepted span is a substring of the original.

Because the output is four small values, a small model can produce it. The chain is us.anthropic.claude-haiku-4-5-20251001-v1:0, then Sonnet 4.5, then Sonnet 4.6, all inference profiles. Haiku leads because it measured faster and, on this fixture, more accurate than Sonnet 4.5, at a median 1.6 seconds against 2.9, which matters for something running once per utterance in a live room. Every ledger row records which route decided it, the model id or the fallback reason, in words under the quote rather than by colour.

Nothing the classifier produces leaves the product on its own. Items land as pending and confirm_item is the only path by which a Bee todo can exist. A rejection is kept as a Correction and shown to the classifier on later utterances in the same course, so the thing that corrects the model is the student's own judgement about their own class.

The consent record is the other half. A lecture is already outside California Penal Code 632 under the public-gathering exclusion in 632(c), so this announces for a different reason: the risk to a student is an instructor who finds out sideways and has the accommodation narrowed. The announcement becomes entry zero of a SHA-256 hash chain, the one in this set of builds, because a disability services officer or a grade appeal committee is an adversarial reader and a record that could have been written afterwards is worth nothing to them. Editing the announcement, backdating it, deleting an item or reordering entries all break the chain, and verify_consent_record names the entry where it stops adding up. Sessions open against policy templates that supply the wording, the expected speaker roles and the retention days, because 28 CFR 35.160(b)(2) leaves the accommodation decision with the institution. The instructor's view returns confirmed announcements with no student name, no roster and no session id, because RoomDigest has no field for any of them.

TypeScript on Node, shipped as an MCP server on @modelcontextprotocol/sdk, with the demo, the review screen and the test suite all calling the same functions so there is one code path. The surface is set as a ruled ledger page: a left margin rail, numbered entries, the consent record at the top because it has to be read first. The test suite passes with no AWS credentials at all.

Challenges we ran into

A lecture is exactly the traffic shape Bedrock throttles. Twenty-one utterances run serially, with no burst, hit ThrottlingException on roughly one call in four, and the response carries no Retry-After and no quota name, so correct backoff is guesswork. The timeout became a budget for the whole call with one jittered retry inside it, and the ledger row records bedrock_throttled so the fallback is visible rather than silent.

Asking for character offsets worked and the offsets were wrong. Both Haiku 4.5 and Sonnet 4.5 landed a few characters off, mid-word, on spans like "rt: I am not holding office hours". Models count characters badly. The span is now snapped outward to the sentences it touches, never inward, so a bad offset costs a slightly longer quote rather than a mangled one, with a fuzz test asserting every accepted span is a substring of the original.

The demand evidence is the weakest part of this. Across 1,692 archived posts from eleven subreddits, hand-coded down to 34 on-topic, zero were about a missed announcement costing someone something, and fifteen were people hunting for a transcription tool, which is the product this deliberately is not. The case rests on the case law and on three studies, and those studies are why it captures less rather than more.

Accomplishments that we're proud of

On the seventeen hand-labelled sentences in the fixture, Bedrock scores 17/17 and the keyword rules score 10/17, missing three announcements with no keyword in them and producing four false positives. Seventeen labels show the shape of that difference and are nowhere near an evaluation, which the demo says on screen. Only one of those two numbers is guaranteed to reproduce: the rules are deterministic and score 10/17 on every run, and Bedrock's 17/17 is a single measurement that could come back different on a rerun, which is a limit of the fixture, not something either score hides.

What we learned

"We told the model not to summarise" is a prompt, and a prompt is a request. If the design principle is that the model may never write a word the student reads, that has to rest on a type system and a validator.

What's next

A live Bee device once there is a way past bee login without the iOS app, real institutional policy wording in place of our four templates, and a co-signed head hash so the certificate can claim more than it does today.

Built With

• amazon-bedrock
• aws-sdk
• model-context-protocol
• node.js
• typescript