So we're running a crypto derivatives exchange and we had a problem to push massive data to clients via the websocket. Since internally we are using Redis, we thought of building an executable where we can subscribe to a Redis pubsub channel and publish to every client that subscribes to a websocket endpoint. You can see the results here https://api.makansepatu.com/#websocket-public-api. By using Redis pubsub we remove the possibility of a backpressure from websocket pushes.
What it does
It publishes data from Redis pubsub channel and to every client that subscribes to a websocket endpoint.
How I built it
We hacked in 48 hours a C++ code and collaborated with Alex Hultman the maker of uWebsockets.
Challenges I ran into
Determining a stateless megaphone.
Accomplishments that I'm proud of
It's written in C++ and eases websocket development internally. We then decided to open-source it so other people could also use it.
We use it in production for our crypto exchange. Our hopes is that if you have a need to blast data from cache to websocket rapidly, just use Megaphone!
What I learned
I learned how memory back pressure is handled.
What's next for Megaphone
Write a proper documentation to allow others to use it easily and make a Python connector.
Install websocat first. Do it by following the installation directions in https://github.com/vi/websocat
There are multiple options for installing WebSocat. From easy to hard:
If you're on Linux Debian or Ubuntu (or other dpkg-based), try downloading a pre-build deb package from GitHub releases and install from GUI or with command like gdebi websocat_..._.deb If you're on Fedora, you can install WebSocat from Copr: sudo dnf copr enable atim/websocat -y && sudo dnf install websocat If you're on FreeBSD, you may install WebSocat with the following command: pkg install websocat. If you're on Mac, you can do brew install websocat. Install the Rust toolchain and do cargo install --features=ssl websocat. If something fails with a -sys crate, try without --features=ssl;
Then run on your terminal
websocat -E "wss://api.makansepatu.com/ws/public/ticker"
websocat -E "wss://api.makansepatu.com/ws/public/trades"
websocat -E "wss://api.makansepatu.com/ws/public/depth/all"