Fixly - Empowering Construction with AI

Upload an image safely, quickly, and easily. Perfect for everyday construction workers
Tells you exactly what the issue is. Gives steps and safety hazards. Everything people use day to day without any brain power

## Inspiration

The inspiration for Fixly came from the realization that while the digital world has advanced rapidly, the physical frontline—construction sites, utility maintenance, and industrial zones—is often left behind. Workers on the ground face high-pressure, dangerous situations where every second counts, yet they are often forced to rely on manual reporting and slow communication chains. We wanted to build a "safety partner" that sits in their pocket, using the latest in Multimodal AI to instantly bridge the gap between seeing a problem and knowing exactly how to fix it safely.

## What it does

Fixly is an AI-native platform that allows frontline workers to analyze and resolve construction hazards "in a snap".

Instant Hazard Analysis: Users upload a photo of a hazard, and the system instantly identifies structural risks, safety violations, and OSHA compliance issues.
Spatial Vision AI: The app automatically draws "Danger Zone" bounding boxes over the photo to visualize exactly where the risk is located.
Detailed Assessment Logs: It generates a clinical "AI Inspector Assessment Log" describing the failure in technical detail, such as a high-pressure water main rupture with high-velocity jets.
Actionable Remediation: It provides a "Worker Summary" with clear, step-by-step instructions (e.g., "Immediately isolate and shut down the water supply").
Auto Procurement: The system automatically generates a "Materials & Tooling" list needed for the specific fix, such as replacement pipe sections and mechanical couplings.
Historical Audit Logs: All inspections are saved to a clinical, minimal "Project Dashboard" that tracks total audits and compliance history across different site zones.

## How we built it

We utilized a high-performance, modern stack to ensure the app was fast and reliable for field use:

Frontend: Built with Next.js 15 and Tailwind CSS for a responsive, mobile-first experience. We used Shadcn/UI to maintain a clinical, high-contrast "Industrial" aesthetic.
Backend: Powered by Appwrite, which handles our secure database for audit logs, image storage in buckets, and user authentication.
Intelligence: The core engine is Google Gemini 3 Flash. We leveraged its multimodal capabilities to process high-resolution images and return structured JSON containing both the technical assessment and the normalized coordinates for our safety bounding boxes.
Logic: We implemented server-side actions to bridge the gap between the Appwrite storage and the Gemini API, ensuring that sensitive site data is processed securely and efficiently.

## Challenges we ran into

One of the primary technical hurdles was achieving accurate spatial reasoning. Getting the AI to not just identify a "pipe leak" but to return the exact pixel coordinates for a bounding box that perfectly encapsulates the "Recorded Hazard" required significant prompt engineering and coordinate normalization. Additionally, ensuring the app functioned smoothly on mobile devices with high-resolution site photos meant we had to implement robust client-side image compression before uploading to our Appwrite buckets.

## Accomplishments that we're proud of

We are incredibly proud of the Zero-Friction UI. A worker can go from seeing a critical water main rupture to having a full remediation plan and a shopping list in under 10 seconds. We also successfully built a "Security Archive" system that feels both professional and authoritative, giving managers a "bird's-eye view" of site safety that was previously impossible without manual, paper-heavy audits.

## What we learned

This project taught us the power of Agentic AI in physical environments. We learned that for AI to be useful on the frontline, it can't just be a "chatbot"—it needs to be a tool that produces structured, actionable data like material lists and safety perimeters. We also deepened our understanding of integrating BaaS (Backend-as-a-Service) like Appwrite with high-speed LLMs to create a seamless real-time experience.

## What's next for Fixly - Empowering Construction with AI

The next step for Fixly is moving from reactive reporting to proactive prevention.

Live AR Overlays: Using Augmented Reality to project the AI's "Danger Zones" onto the worker's camera view in real-time.
Voice-First Audits: Allowing workers to narrate their findings and having the AI combine the audio and visual data for even deeper site assessments.
Direct Vendor Integration: Connecting the "Materials & Tooling" list directly to local supply chain APIs so parts are ordered and shipped the moment a hazard is detected.