SneakersAPI.dev

SneakersAPI is a public sneakers and fashion database API, backed by data from biggest retailers and resellers such as StockX and Nike. As of today, we collected other 1.1 million product codes.

Comment

Inspiration

I was an old reseller, thought I don't do it anymore today. I noticed there was no public database that provides analytics and API to access the data for free. I took the opportunity.

What it does

We mostly support four features:

  • Products and searching API
  • Daily price history, by product and by variant, backed by an API
  • Weekly sales amount API
  • GTIN and Model Number Lookup API

How I built it

It's built with Go and Fiber, previously made in Rust with Actix-Web and a ClickHouse database. I switched to Go, Postgres and DBT pipeline for simplicity and with good indexing it's still very fast.

Challenges I ran into

The data pipeline was clearly the biggest challenge and the one I spent the most time on it. It's actually a combination of many little things, like: being able to link products from multiple sources to get the most accurate date, to this efficiently and multiple time a day without having too expensive hardware and lastly make it fast enough for the API and end-users. I think I rewrote the data pipeline three or four times with many different technologies before finding the good one.

Accomplishments that I'm proud of

Definitely the data pipeline, once again. That took time, but I'm proud of how it came out. It's fast, I run it multiple times a day and it doesn't slow down the API, it's also cheap to host. It could technically run on a 4GB VPS but for the production system I use a biggest VPS.

What I learned

I already had knowledge in data pipeline, backend and frontend. I pushed my knowledge of data pipeline and engineering further. But what was also challenging is the sysadmin/devops side of the project. I have to maintain a 99% uptime, it's part of your ToS and deploying without causing downtime is always challenging, I'm definitely improving and still working on this.

What's next for SneakersAPI.dev

Include more sources, manage even more data and I wish new APIs soon! If I can include daily sales stats I would be more than happy!

Built With

Share this project:

Updates