Inspiration
Temperature problems are quiet until they’re expensive: a server room that creeps up a degree an hour, a cold chain that fails overnight, a sensor that simply stops reporting. We wanted to build the part of that system that has to stay reliable under load: ingest a thousand sensors every second, notice trouble early, and show it clearly, all without dropping a reading. [Add one sentence of your own motivation here.]
What it does
Sapper (صابر) monitors a fleet of simulated temperature sensors in real time. Each sensor reports once per second, and the system:
Detects three kinds of problems: overheating (70°C or more), rapid rise (8°C or more within about 5 seconds), and offline sensors (silent for more than 5 seconds). Dead sensors are as dangerous as hot ones. Warns early. In our heatwave test, the rapid-rise alert fired about 15 seconds before a plain 70°C threshold would have. Shows everything live on a dashboard: a heatmap of all sensors, an alerts feed, a stats panel (messages per second, latency, writes per second, online vs offline), and a history chart per sensor. [Confirm this matches what’s built.] Lets you break things on purpose. A control panel triggers a heatwave in one zone, kills random sensors, revives them, or scales the fleet up to 10,000, so you can watch the system react live. How we built it Simulator (Rust): one MQTT connection publishes for every sensor (sensors//temp), with random-walk temperatures. It also listens on a control topic for the dashboard’s commands. Backend (NestJS): subscribes once to sensors/+/temp, keeps each sensor’s latest state in memory for detection, and pushes a compact snapshot of every sensor to the dashboard once per second. Alerts are batched into a single event per second. Storage (TimescaleDB): readings are buffered and written in one query per second. The database stores one reading per sensor every 5 seconds (the dashboard still gets every reading live), in 30-minute chunks with a 1-hour retention policy, so the database size stays capped. Dashboard (Next.js): a canvas-based heatmap, so 10,000 cells don’t become 10,000 DOM elements. One command: docker compose up starts the broker, database, backend, simulator, and dashboard. Challenges we ran into The database grew by about 1 GB in minutes. At 10,000 sensors, storing every reading cost roughly 100 bytes each. We measured it, then fixed it with 1-in-5 storage, a leaner index, small time chunks, and aggressive retention. Measured growth is now about 98 KB/s at 10,000 sensors, and retention caps the total. Alert bursts. A zone-wide heatwave created 1,000 separate events in the same instant. We batched them into one event per second. Duplicate processing. A service registered in two modules ran twice, which showed up as every log line printing twice and 2,000 messages/sec. We caught it because we were watching the numbers. Threshold order bug. Checking the 60°C warning before the 70°C overheat meant overheat alerts never fired. Docker networking. Inside a container localhost isn’t your PC, and one stray shell command in a SQL init file made the database crash on startup. [Add the DNS issue here only if you solved it.] Learning on the clock. None of us knew MQTT or much Rust before this week. Accomplishments that we’re proud of The backend ingested 10,000 messages per second with the simulator, broker, database, and backend all on one laptop. At 1,000 sensors, end-to-end latency stayed between 22 and 51 ms. [At 10,000, our latency number includes the simulator’s own publish time, so we don’t quote it.] Writing 10,000 readings per second into TimescaleDB in batches, with nothing lost, before we reduced storage to 1-in-5 to save space. A real early-warning result (15 seconds ahead of the threshold), measured rather than claimed. A project anyone can run with a single command. What we learned How MQTT works: publish/subscribe, topics and wildcards, and why QoS 0 is right for once-per-second readings. React to changes, not states: a sensor stuck at 75°C should raise one alert, not sixty per minute. Batch everything that touches the database, and measure disk growth instead of guessing. Rust basics, and how far a single well-written async program goes on a modest machine. What’s next for صابر (Sapper) Smarter detection: compare each sensor against its zone neighbors to catch slow drifts a threshold can’t. Real hardware: replace the simulator with real sensors (for example ESP32 boards) publishing to the same topics. Security: authentication on the control endpoint and the MQTT broker before any public deployment. Notifications: Telegram or email alerts, plus alert acknowledgment. Compression and long-term history: compress old chunks and keep downsampled averages for longer.
Built With
- docker
- javascript
- mosquitto
- mqtt
- nestjs
- nextjs
- postgresql
- react
- rust
- socket.io
- timescaledb
- tokio
- ts