FixPalAI

FixPalAI

Inspiration

This winter, our heater broke down. The technician who showed up spent several hours trying to diagnose the issue. He couldn't figure it out, so he called his supervisor. The supervisor wasn't sure either and had to call someone else. Meanwhile, the technician was flipping through a massive manual, trying to find the right section. After calling 2-3 different people and referencing multiple manuals, he finally figured out it was a simple igniter problem that took 10 minutes to fix. That experience stuck with us. We realized technicians are expected to be experts on hundreds of different systems and thousands of potential issues, but nobody can memorize all of that. They need help, but calling supervisors and searching through manuals wastes time and frustrates customers. We knew AI could solve this problem immediately.

What it does

FixPalAI is an intelligent home repair assistant that gives field technicians instant expert guidance. When a technician encounters an issue, they can describe the problem or upload a photo of the broken equipment. Our system uses specialized AI agents for different domains - plumbing, electrical, HVAC, and carpentry - to analyze the issue and provide step-by-step repair instructions. The system searches through all the ingested repair manuals using RAG to find the exact procedure needed, highlights critical safety warnings, and shows what tools and parts are required. Instead of calling supervisors or spending time searching through manuals, technicians get expert-level guidance in seconds.

How we built it

We built FixPalAI using a multi-agent RAG architecture powered by LangChain for agent orchestration. The system uses specialized AI agents for each repair domain, with anthropic/claude-opus-4-5 via Dedalus API handling the reasoning and response generation. When a user submits a query, we first use the coordinator agent to route it to the appropriate specialist agent based on the domain. For image-based queries, we integrated the Gemini Vision API to analyze photos of broken equipment and identify the specific issue. Each specialist agent has access to a vector database containing embeddings of repair manuals and technical documentation, which we query using LangChain for retrieval. The backend is built in Python, while the frontend uses both Streamlit for rapid prototyping and React with Tailwind CSS for the production interface. We process and store repair manuals and maintain embeddings in a vector database for fast semantic search. The entire system is designed to work quickly, even with complex multi-agent coordination.

Challenges we ran into

The biggest challenge was routing queries to the correct specialist agent. Initially, our system would sometimes misclassify issues - sending an electrical problem to the HVAC agent or vice versa. This led to irrelevant or incorrect advice. We solved this by implementing a smarter routing layer that analyzes multiple features of the query and uses confidence scoring to pick the right agent. We also added a fallback mechanism where, if the chosen agent has low confidence, it can consult other agents. Another major challenge was integrating the Gemini Vision API for image analysis. Getting the API to consistently identify equipment and diagnose issues from photos took significant iteration. We had to fine-tune our prompts and handle cases where the image quality was poor or the equipment wasn't clearly visible.

Accomplishments that we're proud of

We're most proud of successfully implementing vision analysis that actually works in real conditions. Our system can identify equipment from photos and provide accurate diagnostics. This was technically challenging but essential for field use. We're also proud of building a sophisticated multi-agent architecture that genuinely improves response quality. Each specialist agent has deep domain knowledge, and they coordinate seamlessly to provide coherent, accurate repair guidance. This isn't just a simple chatbot - it's a system with real specialization that matches how human experts work.

What we learned

This project taught us how to build and orchestrate multi-agent AI systems effectively. We learned that having specialized agents for different domains significantly improves response quality compared to a single general-purpose agent, but coordination between agents is complex and requires careful design. We gained hands-on experience with the Dedalus API and saw how it simplifies multi-agent orchestration and authentication, which would have been much harder to build from scratch. We also learned how to effectively use the Gemini Vision API for real-world image analysis tasks. Beyond the technical aspects, we learned about the business and enterprise impact of AI tools. We learned to think about ROI, user workflows, and how to position a product for B2B sales rather than just building cool technology.

What's next for FixPalAI

Our immediate priority is adding a parts search and recommendation feature. When the system identifies what needs to be replaced, it will automatically search across multiple suppliers like Grainger, Home Depot, and Amazon to find the best prices and availability. This will save technicians from having to manually look up parts and will help them find the cheapest options quickly. We also want to add a training module for technicians. New hires could use FixPalAI to learn common repair procedures, with the AI guiding them through their first repairs and explaining the reasoning behind each step. This would accelerate onboarding and reduce the burden on senior technicians who currently have to train newcomers. Longer term, we're exploring integrations with inventory management systems so technicians know if they already have the needed parts, and we want to add predictive maintenance features that can identify when equipment is likely to fail soon based on symptoms.