LLM Energy Saver

LLM Energy Saver Logo

Inspiration

We decided we wanted to make the greatest impact for one of the UN Sustainable Development goals. While researching and exploring AI, members of our team realized that AI is incredibly energy-intensive, and AI could help the world even more if this massive consumption was easily reduced with a few clicks on our website. That's how we came up with LLM Energy Saver!

What it does

Our app takes any of your queries and creates an energy-efficient LLM to assist in that specific area. We achieve this by using an LLM to generate multiple queries and scrape the data from the web, then, our system uses that data to fine-tune a small model and convert it into an expert in your specified domain.

By doing this, AI and LLM inference is reduced greatly at the point-of-sale. Larger models are also more energy-intensive, and a custom fine-tuned small model reduces this need for the average user.

How we built it

Our design was focused as a pipeline to the end user. Different members of our team worked on different parts of the codebase! Danjie worked on code to create searches and generate Question-Answer pairs as training data. He also worked on the front end. Andy experimented with different Large Language Models, and set up backend-routing for our API server. Andy also loves writing, so made the presentation and written components. Barry wrote the scraping code and focused on LLM finetuning and creative experimental features. Barry also pushed us through a late night Devpost session!

Accomplishments that we're proud of

We are not afraid to dive deep. Instead of dipping around generating AI using APIs, we go straight into the architectures and the training data.

Challenges faced

Lack of sleep due to waking up early has reduced our energy level, but our passion kept us going. Barry: My biggest challenge was trying out things with a high chance that it might not work. For example, I spent hours trying to fix one bug in the experimental package for quantizing models. During that, I kept thinking maybe I should give up, but I figured it our eventually, and now we have something special in our project.

What we learned

We learned a lot about web scraping, fine-tuning LLMs, and the limits of current open-source models.

What's next for LLM Energy Saver

We are hoping that perhaps LLM Energy Saver could be utilized by many users to actually make an impact towards the 17 UN SDGs. The average user doesn't need a giant-model, but rather information on a specific query during a specific session.

We also seek to integrate new experimental features.

Check out our presentation: https://docs.google.com/presentation/d/1jkWLmmelSFXhgllUaq4eB8bTm9NHCSBpYRNkfGB3nxg/edit?usp=sharing