RePair

What it Does

RePair is a Gemini-powered AI home assistant that analyzes images of household damage and provides instant insights on severity and repair options. It allows users to choose between DIY repairs or professional help, offering hands-free guidance through ElevenLabs voice instructions, recommending parts with prices and purchase links, and matching users with local contractors using an ARM-powered ML model.

How We Built It

The frontend is built with React and Tailwind, ensuring a smooth and responsive user experience. The backend uses FastAPI to handle Gemini API calls and integrate real-time voice functionality with ElevenLabs. Damage analysis and repair insights are powered by Google Gemini 2.5 Flash, while the ARM-based ML model enables efficient contractor recommendations and location-based matching. Real-time voice instructions are delivered via ElevenLabs TTS, creating a seamless hands-free experience.

Challenges We Ran Into

Integrating multiple APIs smoothly in real time was a major challenge, along with optimizing speech recognition for live voice triggers. Parsing structured JSON responses from the AI reliably and coordinating asynchronous processes between image analysis, voice output, and user input required careful design and testing.

Accomplishments We're Proud Of

In under 24 hours, we built a fully functional conversational repair assistant that integrates Gemini Vision, ElevenLabs, and ARM ML into a single cohesive workflow. The system delivers a hands-free repair experience that feels natural, practical, and genuinely useful.

What We Learned

Through this project, we learned how to combine LLMs with multimodal inputs for real-world applications, leverage voice and image synergy for accessible user experiences, and optimize inference and hosting on ARM architecture. We also gained experience building AI systems that are both technically impressive and truly practical.

What’s Next for RePair

We plan to integrate real contractor networks and dynamic pricing APIs, add AR-guided DIY repair instructions, launch a mobile version with offline voice and image capabilities, and expand the system to handle appliance and plumbing diagnostics using sensor data.

Built With

elevenlabs
fastapi
gemini
google-places
python
react
tailwind
typescript

Submitted to

HackNC 2025
- Winner [MLH] Best Use of Arm
- Winner Lowe’s Future of Smart Home Improvement

Created by

I worked on integrating the AI and voice components of RePair. Specifically, I set up the Gemini image analyzer for damage detection, implemented text-to-speech and speech-to-text using ElevenLabs, and connected these features through FastAPI backend routes. I also handled frontend integration to ensure the analyzed results and voice responses were displayed and updated seamlessly for users.

Sagnik Saha
Computer Science @ NCSU
I contributed significantly to developing the backend of our hackathon project by integrating the core AI and data retrieval systems that powered our app’s intelligent features. I built the backend architecture around a LangChain-powered agentic chatbot, enabling dynamic conversations and reasoning for the user’s product inquiries and repair guidance. I implemented API integrations using SerpApi to fetch real-time data on materials, tools, and products, ensuring accurate and up-to-date results for the ordering system.

Farhad Bhatti
For my repair project, I developed the backend logic to find nearby contractors based on a user’s location and integrated frontend features to create a seamless user experience. I also used ARM and Google Cloud to train and deploy a machine learning model that recommends the most suitable type of contractor based on the user’s repair needs, leveraging cloud infrastructure for scalable performance and real-time predictions.

Adith Venkatesh
I contributed significantly to developing the frontend of our hackathon project by building key React components such as the Order page, Diagnose and Repair modules, Sidebar navigation, and the Home landing page. I implemented dynamic features like the material ordering system with chat-based product search and AI assistant interaction, an intuitive damage diagnosis upload and analysis interface, and a voice-activated repair assistant with step-by-step guidance and contractor search. I also developed the Sidebar for tab navigation with unlockable workflow steps, and added user experience improvements including a dark mode toggle and animated backgrounds. Throughout, I focused on creating a user-friendly, visually appealing UI with responsive layouts and smooth transitions to enhance the overall app experience.

Soumil Kothari

Updates

Sagnik Saha started this project — Oct 12, 2025 10:38 AM EDT

Leave feedback in the comments!

Log in or sign up for Devpost to join the conversation.