Inspiration
One in four U.S. adults lives with a disability, yet the tools we work in every day weren't built for them. A blind teammate's screen reader hits a screenshot and goes silent — just "image," then nothing. A neurodivergent or non-native-English teammate opens a thread and finds a wall of acronyms; 54% of U.S. adults read below a sixth-grade level, and most workplace writing lands far above it. Slack is where work happens now — and for these colleagues, half of it is a closed door. We wanted the fix to live inside Slack, and to be impossible to argue with: every fix had to come with a number, not a claim.
What it does
gurtYos is an accessibility co-pilot that makes Slack usable for the people it currently excludes:
- Alt text on demand — react 👁️ on any image and the agent writes screen-reader-quality alt text in-thread (Claude vision). Post an image without alt text and it offers unprompted.
- Plain-language rewrites — react 🧩 on a jargon thread and it rewrites it with a measured reading-grade before → after (grade 23 → 4 in our demo), plus concrete deltas: acronyms defined, sentences split, reading time saved. React 👎 and it re-renders simpler — and remembers your level.
- "Catch me up, accessibly" — ask the Assistant and it produces a screen-reader-friendly canvas digest of a channel, in your language, while the agent's audit passes stream live so you watch it check its own work.
- Channel accessibility report — one question scores a whole channel (ADA / Section 508 framing, score 26 → 97 projected), and one click — Fix this channel — applies alt text and rewrites across the channel.
- Natural-language preferences — "now in Spanish", "set my reading level to 5" — parsed, persisted, applied to everything after.
How we built it
Three required Slack platform technologies, each load-bearing — remove any one and a feature breaks:
- Slack Assistant API — the co-pilot surface, with live plan/task streaming so the agent's audit passes render as visible steps.
- A custom MCP server (FastMCP, Python) — a deterministic accessibility scorer: Flesch–Kincaid reading grade, WCAG contrast, jargon and long-sentence audits. It drives the agent, not decorates it.
- Real-Time Search (
assistant.search.context) — powers topical and cross-channel catch-up.
The heart of the system is a draft → audit → revise loop: the agent writes a draft, calls its own MCP scorer over a live stdio session to grade it, and revises until it meets the reader's target grade. The numbers on screen are computed by deterministic code, not claimed by the model. Around that core: Bolt for Python on Socket Mode, Claude Sonnet 4.6 for digests and Haiku 4.5 for rewrites/alt text, SQLite for user preferences (never Slack content), spend/rate guardrails, prompt-injection fencing for all channel-sourced text, and a fake-Slack-client test harness — 59 offline tests plus gated live integration tests and a headless end-to-end self-test.
Challenges we ran into
- The agent loop deadlocked intermittently. The tool-runner ↔ MCP stdio integration would occasionally hang forever. We isolated it with a dedicated repro script (raw MCP calls were fine; the combination wasn't) and bounded every run with a wall-clock timeout so a hang degrades to a clean fallback instead of a frozen demo.
- Capping the loop broke it in a non-obvious way. Lowering the iteration cap to "speed things up" truncated the agent before it emitted its final answer, producing empty output. The fix was understanding that the cap is a ceiling, not a target — the loop naturally converges in ~2 audit passes.
- Real-Time Search is a search, not a reader. A generic "catch me up"
query starves it. We split retrieval: named-channel catch-up reads
conversations.historydirectly; RTS powers the topical/cross-channel path. Routing each question to the right retrieval was worth more than any prompt. - Slack's own text is untrusted input. Channel messages, threads, and search results all reach the model, so a hostile message ("ignore previous instructions…") — or text inside an image — could hijack the agent. All untrusted content is fenced and spotlighted as data-to-transform, never instructions.
- Socket Mode on macOS fought us twice. The built-in client hit a
BrokenPipe reconnect loop (fixed by switching to the
websocket-clientbackend), and two connected sockets on one app token silently round-robin events — the "exactly one instance" rule is now enforced by a launchd supervisor that keeps the judging instance alive 24/7.
Accomplishments that we're proud of
- All three platform technologies genuinely load-bearing — Assistant, custom MCP, and RTS each carry a shipped flow, not a checkbox.
- An un-fakeable technical signature: the agent grades its own homework with deterministic code, live, and you can watch every pass.
- Impact as a number — reading grade 23 → 4 on a real legal wall of text; a whole channel scored 26 → 97 with one click doing the fixing.
- It practices what it preaches: the App Home, the report canvases, and even our README are themselves fully alt-texted and screen-reader-friendly.
- Production shape: guardrails, injection fencing, 59 offline tests, a headless live self-test, and a keep-alive supervisor for judging week.
What we learned
- Agents need external, deterministic verification. A model asked "is this simpler?" will say yes. A scorer that computes Flesch–Kincaid can't be charmed. The draft → audit → revise loop only works because the auditor isn't the model.
- Retrieval routing beats prompt engineering. Knowing when to search vs. when to read history fixed more real failures than any prompt tweak.
- Accessibility is measurable, and measurement changes the conversation — "this thread is grade 23" lands where "this is hard to read" doesn't.
- On a brand-new platform, build your own repro harnesses early. Every hard bug (the MCP deadlock, the streaming chunk schema, the RTS routing) fell to a small isolated script, not to staring at the app.
What's next for GurtYos
- A living jargon glossary that learns each org's acronyms and feeds the scorer.
- More languages for rewrites and digests, with per-user defaults.
- An org-wide accessibility dashboard — the channel report, rolled up to a workspace compliance view over time.
- Slack Marketplace listing so any workspace can install it.
Log in or sign up for Devpost to join the conversation.