Refurnish AI

homepage

Inspiration

At work, most of May and the beginning of June were spent working on an ecommerce end-to-end feature where users can configure 3d furniture coming from base 3D models that were stored on the backend.

When I saw the hackathon, I wanted to build a generic 3D builder for SMB's where product pictures could be imported and converted to 3D and be fully configurable for ecommerce...this was dropped after some thinking because of the difficulty of implementation in one month and lack of domain knowledge.

After some more thinking, 2 weeks into the competition, I settled on using Stable diffusion to generate Interior designs and finding a way to identify the furniture in the generations and directing users to trusted sellers like IKEA and Amazon.

What it does

For this MVP, I stuck with Living rooms. Users can select the style of living room they want to generate and refurnish will generate a realistic-looking image and then annotate the furniture in the room with real-world product links.

Used IKEA for MVP as they were easily scrapped.

How we built it

Bolt- UI - nextjs
supabase- database
replicate - model service
flux - image gen
SAM - segmentation
CLIP - classification, matching
python backend
AWS EC2 - server

Challenges we ran into

ran into challenges with backend, never used fastapi, GPT helped pull me through
initially planned to host models on aws but too expensive so went with replicate
could not find good models for sd + controlnet on replicate so went with flux
Deployment has been a hassle
started backend late,(over this past weekend, working on it still while submitting)

Accomplishments that we're proud of

it's working

What we learned

most of the tools used to build are new to me so I learned those.

What's next for Refurnish AI

~~get it fully working after submission; little bug with annotation~~, fixed
hope I can change youtube video for same url or I'll pin a new video tot he submitted one in the comments
if app does well in competition, i will see if I can get this to market and incorporate with other brands or via partnership with local brands, as scrapping is not viable for a business.
- this would open up opportunities like training the generation on the exact products from businesses
- tighter integrations into the shopping experience
- potential white labelling
possibly adding AR and 3D features
There could be so much done on this