GameofLife

Loading Screen
Detection
Quest Complete
Chat Window

Inspiration

We got inspired for this project on the train ride here. Given it is a 3 hour ride, we all got a bit antsy and restless. We wondered: What if we were able to make this more interesting? We then thought a game based on real life would be very fun, and here we are.

What it does

After loading into the loading screen, the camera will detect any person in frame. After a second or two, it will assign them a health and mana stat based on features like the size of their bounding box and the color of their clothes. By clicking on someone, you can pull up a detailed page about them including their full title and a short description about them. It also gives you the opportunity to chat with them. On the top right corner, you have a main and side quest long. The main quests are based on your imported calendar and the side quests are generated at random. In order to complete a quest and gain xp, you must submit a photo of you doing the quest and the AI will determine if you have successfully completed it.

How we built it

In order to build it, we used YOLOv8n for the person detection and bounding boxes. We used both the Llama and Gemma models to implement the full title, description, chat, main quest generation, and quest completion features. To power this, we used Featherless and OpenAI.

Challenges we ran into

When implementing the main quest UI and clickable mouse regions, we were completely unfamiliar with utilizing Python for those parts. We also had trouble getting the correct focal point for the HP and mana system, as we were trying to stop the stats from fluctuating based on the person's distance.

Accomplishments that we're proud of

We are super proud of utilizing the OpenAI API and Featherless to turn mundane tasks into game objectives. This was a major goal for us, as was using the AI to generate descriptions for people in different roles. We are also proud of the code that assigns multiple roles to users at once, and especially the chat window feature, which allows us to chat with characters just by clicking on them.

What we learned

First, we learned how to utilize YOLOv8 for object detection. We also gained experience building user interfaces with Python, and learned how to leverage the OpenAI API and Featherless to enable AI analysis of the real world.

What's next for GameofLife

With the rapid advancement of smart glasses, we believe that is how Game of Life can evolve. We want the game to be a seamless part of your daily routine, transforming mundane tasks into interactive and beneficial gameplay. We believe having random side quests like meeting new people and drinking water is a fun and effective way to bridge the gap between the digital world and self improvement.