We were inspired by the growing use of social media amongst our youth and the increasing mental health concerns that extensive use causes. We spent a lot of time retraining our LLM to fine-tune it to our specific needs and to improve its performance. One of the first struggles we faced was optimizing our data pipeline, as it took over 2 minutes to execute per user. We found that the LLM we trained was taking 7s per post or comment, so it took a while for people with 200+ contributions, even with us running 10 containers in parallel. We improved this execution time to 3s by optimizing the code, but that still wasn't good enough. To improve performance further, we trained a second model that was significantly smaller than the first model, and this got our execution time down to < 1s per post or comment.
The most important concepts that we learned from this project were:
- How to coordinate between people working on different aspects of the code
- Only focus on optimizations after you get a working product
- A bigger AI model is not always the best choice
- Communication is one of the most valuable skills to have when working in teams
Log in or sign up for Devpost to join the conversation.