We finally got our excuse to write Elixir and we picked work that lets it stretch: a live feed, lots of fetches at once, and a UI that should move as soon as findings land. Dripstone is our botfarm tracker on Phoenix 1.8 with LiveView, Postgres 17 and Oban at dripstone.dev. Since X has no free path and Reddit closed API access, we built on Bluesky Jetstream for discovery and the AppView API for fetching. When several accounts post the same link in a short window we call it a candidate group, then Oban jobs fetch the members in parallel through one shared Req politeness layer. Each group gets a heuristic score of \(\text{score} = \min(\text{count}/50,1)\cdot 40 + \text{link_ratio}\cdot 30 + \text{regularity}\) with \(\text{regularity} = 30\cdot\max(0, 1 - \sigma_{\Delta t}/\mu_{\Delta t})\), and we let Jev do the classification because it is fast and gives us the same answer for the same posts, so we can keep versioned edges and query by first_seen_at, last_true_at and last_checked_at. Seeing bursts turn into linked networks on the overview screen in real time still feels great.
Our favorite work was making that pipeline cheap in the way Elixir likes. Group scoring is one pass over sorted gaps for mean and variance with no extra traversals, map lookups use lazy fallback so we skip string work on hits, and Jev input is capped at 8k with a 50 post byte budget. The rate gate lives in an owner process with an atomic check-and-set so jobs stop racing and the table stops vanishing. Tracker writes go through a Writer GenServer that turns a whole candidate group into one write and flushes feed health once a minute instead of on every post. That mix of bounded windows, per link cooldowns, hourly caps, and PubSub diffs to LiveView is what keeps Bluesky, our DB, and our Jev bill calm.
Links were just the first signal because they are easy to see, and the same shape fits timing sync, repost rings, shared domains, and text similarity. Each check is a small module behind the same fetch and assess shape, so we can add a new one without touching the rest, and Elixir releases plus small supervised processes mean we can ship that with barely a blip. That is also how we think about the shared store: every finding adds to one relation graph, and we prune one offs and stale windows as time passes so the graph stays light and the accounts that keep showing up together stay visible. We love that the system grows by adding small pieces instead of rewrites.
Built With
- bluesky
- elixir
- jev
- linux
- mise
- phoenix
- postgresql