Floor Plan Furnisher AI

Teammates and Roles

Kajol Prasad - did a lot of the writing and proofreading for this Hackathon and helped create prompts for the AI model. I did most of the training for the GPT builder model.

Pranav Yadav - Helped with training the GPT model and creating floor plans to test. Helped write part of the final document and edited the final youtube video.

Amir Walker - Helped organize the entire hackathon project and streamlined the different ideas we had in the beginning. Helped troubleshoot the ChatGPT API and knows Python.

Inspiration

One of the biggest issues our generation faces today is the housing crisis and the rise of rent/house prices. As Freshmen in college ourselves, one of the most uncomfortable changes in our lives was the shift from living in our parents’ homes to sharing a small room with multiple strangers. Many of us know that in order to be able to afford a home in the future, we may need to live this way for multiple years even after college and save up our money. Although our AI model doesn’t tackle every issue with housing, we wanted the social aspect of our project to help normalize the idea of living with roommates for a lot of our adulthood. Our project puts accessibility to all incomes first with a budget feature and prioritizes splitting space evenly for each roommate. This means that even if one roommate has a fridge, bed, and a large desk while another roommate only has a bed, they will still get the same amount of allocated space. We’ve all had our fair share of roommates that want to split space unevenly, so we felt it was important for our AI to keep everything equal.

What it does

Our AI model takes in 5 inputs: An image of a floor plan with measurements, number of roommates, pre-owned items that must be added to the floor plan, desired items to be added to the floor plan, and a budget. The AI will firstly allocate equal spaces for each roommate and will create a walkway starting from the door that leads to each added furniture item. The model will then output the floorplan image with a visual Matplotlib overlay that represents each item added. Along with this, the model will output the total cost estimate of all added items and 2 yearly total electricity cost estimates: one with energy-efficient appliances, and one without. Although not implemented, our idea also includes a Nextdoor marketplace API which will allow our AI model to recommend pre-owned furniture and appliances to the user in order to save costs and promote sustainability.

How we built it

We first attempted to use the ChatGPT API to build our AI model which ended up being too complex for us as we couldn’t figure out how to input images and get an image output. ChatGPT's GPT builder was a more user-friendly solution for us as we have little experience with AI and coding and this required no code. In the configuration page of the custom model, we added prompts and a reference image of common floor plan symbols such as doors, windows, and outlets. Rather than creating a new image with DALL-E or editing the image directly, we opted to use Python’s Matplotlib in order to create an overlay on top of the original floorplan of the desired items. Fine-tuning this process took a lot of time as it was a process of providing an input, troubleshooting the output, and adding an adjustment prompt in the GPT builder. We also predetermined the logic we would use which is highlighted in our PDF document, but our logic put fair space allocation, placing of essentials such as a bed, creating a walkway, and sticking to the user budget at the forefront.

Challenges we ran into

We ran into many challenges, the first being with the ChatGPT API. We ran the API through repl.it and attempted to feed it prompts manually through Python code. Although we did figure out how to input an image, we weren’t able to edit that image to create the desired output. After switching to the custom GPT builder, we had issues using DALL-E as its outputs were often random colorized and 3d images that didn’t follow the user inputs. We switched to Matplotlib which worked better but would still clip items through walls and had subpar wall, door, and window detection.

Accomplishments that we're proud of

We are proud of our ability to work together as a team doing our first hackathon and solve problems that we had during the entire process. As none of us have experience with AI and are beginners to coding, finishing this project required a lot of learning and collaborating which we were able to do. We are additionally proud of choosing a social topic that is personal to us and creating something that helps normalize living with roommates. Our biggest accomplishment during this hackathon was going from little knowledge about AI to creating a (partially) working AI model.

What we learned

We learned how to create and train our own Custom GPT geared towards our specific use case; the model was trained with pictures consisting of standardized symbols and images it could recognize. We also learned how to efficiently train an AI using prompts and examples and learned the difference between ChatGPT’s DALL-E and Python’s Matplotlib in image generating and editing. Before this project, none of us knew much about API’s, but we have gained valuable knowledge on API’s real-time capabilities and communication with front-end programs.

What's next for Floor Plan Furnisher AI

Training the model for a cleaner floor plan generation is the top priority for our project. Due to its challenges with recognizing walls in our floor plan images, further training is needed with more images and feedback. We can also improve the suggestions portion of our chatbot, fine-tuning its suggestions to the user about possible items they could buy, as well as explaining its decisions in the layout. We now know that AI models improve over time given multiple examples, so we will continue to train the AI model with images of floorplans that highlight and label the walls, windows, doors, and outlets. With more time, we can also successfully implement our use of the Nextdoor API, allowing our GPT to access market listings in the user’s area on the Nextdoor app and create our own frontend website to implement both Nextdoor and Chatgpt’s API.