Inspiration
I always appreciated great interior design, so combining that with my passion for AI and Computer Science urged me to create a tool that makes it easy for house builders or architects to quickly draft and visualize their plans.
What it does
Simply takes in a prompt from a user, and produces a 3D room using the assets from www.kenney.nl and Google Gemini.
How I built it
I used Python as my chosen programming language, mainly due to my fluency in it. I also used Panda3D to load in the models and visualize. I used the google.generativeai library to access the Gemini API.
Challenges we ran into
The most challenging part was definitely the prompt engineering. It was really difficult to consistently get accurate and efficient results, and combined with how little Gemini knew about the exact details of the assets it used, it often got rotations incorrect.
And like every other LLM, Gemini does mess up in comprehending the exact positional instructions of each prompt, and placement of objects can be random and inconsistent. However I think I've managed it well compared to when I first realized its flaws for the first time.
What we learned
It's best to fine tune models instead, but that would need extensive amounts of training data that I just didn't have. Keeping instructions as concise as possible was key to prompt engineering. For me, this project was also a good exercise for decomposition, a key method of computational thinking.
What's next for DesignMyRoom
Findings ways to improve accuracy and consistency is a must. The asset library must also be expanded as it's quite limited, and the AI would ideally get more flexibility in terms of the actual room layout (shape, size of rooms, floorings).
Log in or sign up for Devpost to join the conversation.