Super scalable Stock API

Inspiration

I always enjoyed thinking about scalability and systems design and one of the main reasons this needs to be done is to process lots of requests for, for example, data. So I decided to build a simple API that processed data, but make it able to handle hundreds of requests a second.

What it does

It takes a request in the format of

{
    "stock_name": "MSFT",
    "price": 60
}

And 1) saves it to prometheus for short term stats 2) saves it to mongodb with a timestamp for further use later on.

That is it, simple idea but designed to be super scalable

How we built it

Written in Rust to be fast and secure, using Redis, MongoDB, Grafana, Prometheus all running in Kubernetes with Minikube

Challenges we ran into

Had to switch from PostgreSQL to mongodb and from zeroMQ to Redis due to a lot of issues with them.

Accomplishments that we're proud of

To finish! It was a pretty ambitious idea for 1 person but I am glad I finished it

What we learned

ZeroMQ SUCKS

What's next for the project

Hopefully other people can use this proof of concept :)

Built With

• grafana
• kubernetes
• mongodb
• prometheus
• redis
• rust