What it does
LambdaQ is a lightweight, distributed server system that allows developers to autoscale painlessly and cost effectively. By having users defining and deploying just the function they need and the language they are using, this service creates a fault tolerant RabbitMQ based queuing system with multiple workers. The worker pools automatically assign more or less workers to a task based on response time, allowing for load balancing. This project is quite language agnostic, so it can support many programming languages because of powerful templating, unlike similar services that exist presently. In other words, we let the user only concern themselves with the actual functions they want to deploy instead of worrying about setting up a full fledged backend.
Many existing services that accomplish similar tasks are expensive, clunky, and not friendly for many languages. We wanted to change all that.
How We Built It
Using the RabbitMQ service and docker, we set up a frame for users to input functions as a string, templating to autogenerate code for queueing, and a worker scaling system that adjusted its own size.
Challenges we ran into
We had to make sure that our load balancing scaling system would not die out suddenly, and that we would not cause any resource leaks.
What we learned
We learned a lot about the inner workings of distributed systems, as well as functional and metaprogramming.
We're proud of
We are proud of the fact that we were able to create a system that actually works and can be used in the real world by developers.
We have a list of features that we are going to work on beyond the scope of this hackathon, including:
- Allowing users to input multiple functions at a time and specify basic workflow
- Implementing better chaining for tasks
- Improving support for other languages
- Incorporating dependencies that user functions may have
- Database matching (since currently we are working with a local host)