-
-
Collaborating inside trmi: @bck-stack and @talhacomak in the shared voice call and lane approving SAFE EDIT cards.
-
Collaborative Post-Mortem summary recording responders, pinned evidence, quorum commands, and MTTR metrics.
-
Active SEV-1 outage (INC-4021): 60 FPS topology canvas showing webhook retry storms and 64% Postgres pool error rates.
-
Two-engineer Quorum (2/2) executed: live responder cursors, pinned root-cause logs, and topology healed to RESOLVED state.
Inspiration
During a production outage, debugging usually fragments across isolated screens: one engineer tails logs in a terminal, another checks dashboards, people paste cropped screenshots into chat, and someone runs a risky mitigation command without a second pair of eyes. While building together in trmi, we noticed how much safer and faster development felt when our team shared one room, talked on the built-in voice call, and approved high-risk changes via vote cards. We decided to bring that exact multiplayer workflow to live production incident response.
What it does
PulseRoom turns outage triage and architecture debugging into a shared, real-time multiplayer war-room:
- 60 FPS Multiplayer Topology Canvas & Cursor Bubbles (
/): Responders join a shared room (?room=WAR-ROOM-01), see every teammate's live cursor, active tool (Inspect,Trace,Patch), and floating cursor speech bubbles (/hotkey). Dragging any microservice node updates its layout across all connected browsers in real time. - Shared Root-Cause Evidence Pinning: Clicking any service node filters the distributed log stream. When a responder pins a suspicious log line (such as
142 backend processes idle in transactiononwebhook-worker), it immediately lands on the shared Evidence Board with their name and timestamp. - Quorum-Gated Remediation Cards (Two-Key Turn): Risky infrastructure fixes are modeled as shared approval cards across three tiers (
ROUTINErequiring 1 approval,BREAKINGandDECISIONrequiring a 2-engineer quorum). Once the second responder approves a breaking card, PulseRoom executes the fix against the topology engine, draining retry storms, dropping error rates from64.0%to0.2%, and turning packet flow back to healthy green. - Playable Outage Scenarios & Autonomous SRE Peer (
Maya): Includes three production outage scenarios (INC-4021Postgres Connection Pool Exhaustion,INC-4088Flash Sale Cache Stampede, andINC-4104Split-Brain JWT Key Rotation), a one-click+2nd Engineer Windowlauncher for split-screen testing, and an autonomous SRE teammate (Maya) who inspects degraded nodes, pins evidence, and casts quorum votes. - One-Click Collaborative Post-Mortem: Compiles a complete Markdown post-mortem report recording MTTR, responders present, pinned evidence, quorum-approved commands, and the full audit timeline.
How we built it
We built PulseRoom as a zero-dependency Node.js RFC 6455 WebSocket server (server.mjs) and Cloudflare Worker (worker.mjs) sharing a deterministic graph telemetry engine (src/engine.mjs, src/scenarios.mjs) and an HTML5 Canvas front-end (src/ui.mjs).
We coordinated our work inside trmi (@bck-stack and @talhacomak) connected in the shared voice call and parallel lanes (cursor-shortcuts-doc in PR #1 and quorum-metrics-do in PR #2). Using trmi's SAFE EDIT approval cards, we documented our architectural decisions and edge cases directly in .termi/decisions.md and .termi/gotchas.md while testing the live Worker deployment side by side.
Challenges we ran into
- Separating 22 Hz Ephemeral Cursor Frames from Causal Room State (
stateVersion): Broadcasting cursor movement 22 times per second while two engineers simultaneously clickApproveon a remediation card caused unnecessary DOM re-renders when everything shared a single state counter. We split the protocol so cursor frames update only the 60 FPS canvas and presence bar, while causal room mutations (VOTE_CARD,PIN_LOG,MOVE_NODE,PROPOSE_CARD) incrementstateVersionand trigger ordered DOM updates. - Viewport-Normalized Topology Coordinates:
Testing between different laptop DPI scales (
125%vs100%) initially clipped rightmost topology nodes (Redis ClusterandPostgres Primary). We implemented an automatic world-to-viewport transform (getCanvasTransformandscreenToWorld) that normalizes cursor coordinates to a1040x560world space so all 7 microservice nodes fit cleanly on any screen size.
Accomplishments that we're proud of
- Designing a multiplayer state engine where applying a remediation card actually recomputes upstream/downstream packet flow, p99 latency, and error rates in real time across every connected responder's screen.
- Shipping both a zero-dependency local RFC 6455 WebSocket server and a live Cloudflare Worker deployment so anyone can test multi-window quorum voting in one click with zero login friction.
What we learned
- Working inside trmi with parallel lanes and explicit
.termi/decisions.mdand.termi/gotchas.mdfiles prevented merge conflicts when one of us tuned the scenario engine while the other adjusted canvas DPI scaling. - High-frequency presence telemetry (cursors, drag positions) and strict consensus actions (quorum votes) should never share the same state-versioning loop.
What's next for PulseRoom
Adding live OpenTelemetry OTLP log ingestion, Prometheus PromQL metric bindings for real Kubernetes clusters, and voice-to-cursor transcription inside the war-room canvas.
Log in or sign up for Devpost to join the conversation.