FixIt AI

Fix broken tech with just a video. FixIt AI leverages Gemini 3's multimodal vision to identify devices, diagnose failures, and guide your repairs safely

Comment

💡 Inspiration

We've all been there: a broken appliance, a malfunctioning gadget, and no idea where to start. Professional repairs are expensive, and service manuals are dense and technical. We wanted to democratize technical expertise. We realized that with Gemini 3, we could create an AI agent that "sees" what you see—turning a smartphone camera into an expert mechanical engineer that guides you through repairs in real-time.

🤖 What it does

FixIt.AI is a universal visual repair assistant.

  1. Video Diagnosis: You upload a video of your broken device. FixIt.AI uses Gemini 3 to identify the exact make and model, and analyzes visual cues (like flickering lights, strange noises, or physical damage) to diagnose the problem.
  2. Generative UI (Visual Guides): We go beyond text instructions. FixIt.AI uses Gemini 3 to dynamically write code (HTML/SVG) that renders animated visual aids in real-time. If the instruction is "rotate the screw counter-clockwise," the AI generates a custom visual showing that exact action on the fly.
  3. Safety First: A dedicated Safety Agent evaluates the risk. It identifies high-voltage components or dangerous procedures and blocks them if they are unsafe for DIY.
  4. Manual Retrieval: It automatically finds and cross-references official service manuals to ensure accuracy.

🛠️ How we built it

The application is built on a FastAPI backend and a React frontend.

  • The Brain: The core is the gemini-3-flash-preview model. We set up an agentic workflow where Gemini acts as different "experts" (Identification Agent, Diagnosis Agent, Safety Agent).
  • Generative UI Engine: We implemented a unique prompting system where Gemini outputs raw code to visualize physical actions (like screwdriver rotations), which the frontend renders immediately as interactive graphics.
  • Multimodality: We leverage Gemini's native ability to process video files directly. We feed the raw video stream to the model to extract timestamps of failure symptoms.
  • Infrastructure: We used MongoDB to store session history and repair logs, and deployed the frontend on Vercel.

🚧 Challenges we ran into

  • Safety Guardrails: Determining strict safety boundaries was hard. We had to prompt engineer Gemini 3 to act as a strict "Safety Gatekeeper" that refuses to help with microwaves (high voltage capacitor risk), while still being helpful for toaster ovens.
  • Dependency Conflicts: Getting the modern AI stack (Gemini SDKs) to play nice with standard web frameworks required some deep dependency resolution, specifically around google-api-core and strict environment pinning.

🏅 Accomplishments that we're proud of

We seamlessly integrated video analysis into the repair workflow. Seeing the AI correctly identify a specific device model just from a 5-second video clip and then dynamically generating a visual diagram of the repair step was a magic moment.

🧠 What we learned

We learned that Gemini 3 is surprisingly good at "physical common sense." It understands mechanical relationships (e.g., "if this screw removes the cover, the battery is underneath") better than purely text-based models. We also learned that Gemini is an excellent UI designer, capable of coding functional visual aids on demand.

🚀 What's next for FixIt.AI

  • AR Overlay: We plan to implement real-time AR overlays that draw circles around the specific screw or cable you need to touch on your camera feed.
  • Part Ordering: Integrating with APIs to automatically find the replacement part online once the diagnosis is confirmed.

Built With

Share this project:

Updates