Sahay The Hardware Repair Agent

Sahay is a multimodal AI agent that identifies electronic components and provides real-time repair guidance through the lens of your smartphone.]

Comment

Sahay: The Multimodal Electronics Repair Agent

Inspiration

As a second-year Mechanical Engineering student at College of Engineering Perumon, I spent many hours in labs surrounded by complex machinery and PCBs. I noticed a recurring problem: when a component fails, the gap between seeing the damage and knowing how to fix it is massive.

Manuals are often hundreds of pages long, and senior mentors aren't always available to guide a student through a repair. My experience as a finalist at the National Students' Space Challenge (NSSC) at IIT Kharagpur taught me that in high-stakes environments—like rocketry or robotics—every second counts. I wanted to build a tool that gives every technician the eyes of an expert engineer, reducing e-waste and downtime through the power of Gemini 3.

What it does

Sahay is an "Agentic AI" assistant that helps users diagnose and repair electronic hardware. By using a smartphone camera, users can identify components, read resistor color codes, and find IC pinouts instantly.

Unlike a simple chatbot, Sahay is an agent: it doesn't just explain the problem; it plans the repair. It retrieves technical specs from massive PDF manuals, calculates circuit values (like the resistance needed for a specific voltage divider), and can even search for and draft a procurement request for replacement parts on local Indian electronics stores like Robu.in or IndiaMART.

How we built it

Sahay is built on a Vision-to-Action architecture designed to bridge the physical and digital worlds:

  • The Brain: I used Gemini 3 Pro for its industry-leading multimodal reasoning. It allows the agent to process live video feeds of circuit boards.
  • Knowledge Retrieval: Using the 1.5M Context Window, I ingested a library of technical datasheets and Mechanics of Solids principles. This allows Sahay to cross-reference a visual component with its theoretical specifications instantly.
  • Agentic Actions: I integrated Google Antigravity to allow Sahay to perform real-world tasks, such as searching for replacement parts or drafting maintenance reports.
  • Logic & Math: The agent uses Python-based tools to calculate circuit parameters. For instance, if a technician needs to replace a resistor in a voltage divider, Sahay calculates the required resistance using: $$V_{out} = V_{in} \cdot \frac{R_2}{R_1 + R_2}$$

Challenges we ran into

  • Visual Complexity: Circuit boards are dense. Differentiating between a diode and a small capacitor in low-light conditions was a challenge. I solved this by implementing a "zoom-and-confirm" loop in the system prompt.
  • Latency in Live Video: Processing high-resolution video frames in real-time for repair guidance required careful optimization of the Gemini Live API calls to ensure the advice stayed in sync with the technician's movements.
  • Hardware Diversity: Every manufacturer has a different design language. I had to focus the agent on identifying standardized markings (like IC part numbers) rather than just shapes.

Accomplishments that we're proud of

  • Zero-Shot Identification: I am proud that Sahay can identify complex SMD components and their functions without any prior training on those specific board layouts.
  • Meaningful Utility: Creating a tool that has the potential to reduce electronic waste in rural workshops across Kerala by making repair more accessible than replacement.
  • Seamless Integration: Successfully combining the 1.5M context window with real-time vision to create a truly "expert" engineer persona.

What we learned

Building Sahay taught me the importance of Agentic AI over traditional chatbots. I learned how to:

  1. Prompt for Spatial Reasoning: Training Gemini to identify tiny components by their coordinates on a PCB.
  2. Context Management: Effectively utilizing a massive context window to ensure the AI doesn't hallucinate pinout diagrams.
  3. Human-AI Collaboration: Designing a workflow where the AI assists the human hand without overcomplicating the repair process.

What's next for Sahay: The Hardware Repair Agent

The future of Sahay involves moving from "assisted repair" to "autonomous diagnostics."

  • Integration with IoT: I plan to connect Sahay to digital multimeters and oscilloscopes via Bluetooth so the agent can "feel" the circuit (voltage/frequency) while it "sees" it.
  • Expansion to Robotics: Bringing Sahay’s reasoning to mechanical systems, such as helping students at CE Perumon debug their line-follower robots or 2-wheel kits.
  • Smart Curve Integration: Using the core vision technology of Sahay to enhance my "Smart Curve Traffic Alert" project, allowing the system to identify vehicle types and predict collision risks with higher accuracy.

Built With

Share this project:

Updates