VisiPillar

Our app uses AI to instantly identify broken CAT machinery from a single field photo. We replace manual reporting with exact part numbers and fitment certainty, slashing downtime.

Comment

Inspiration

Construction workers are the backbone of our built environment — yet their hard work often goes overlooked and underappreciated. This project was born out of a deep admiration for the men and women who show up every day, exposed to scorching heat, heavy rain, freezing snow, choking dust, and countless other harsh conditions, just to get the job done.

My goal was simple: do anything I could to improve their quality of life.

One area ripe for change is the tedious, time-consuming process of manually inspecting and documenting equipment damage. Workers on Caterpillar job sites are currently burdened with paper checklists and clipboard-based reporting — a slow, error-prone process that hasn't kept up with the demands of modern construction. In an age where AI is transforming every industry, there's no reason this workflow can't be revolutionized.

What it does

VisiPillar is an AI-powered damage recognition and reporting tool designed specifically for Caterpillar construction equipment. Workers simply point their camera at a piece of machinery, and VisiPillar automatically:

  • Detects and identifies visible damage — including dents, cracks, rust, broken components, and wear patterns
  • Generates a structured damage report in seconds, complete with damage type, severity, and location on the equipment
  • Logs reports digitally so they can be instantly shared with supervisors, maintenance teams, or insurance providers
  • Reduces human error by replacing subjective manual assessments with consistent, AI-driven analysis

No more clipboards. No more missed damage. No more delays.

How we built it

VisiPillar was built using a combination of modern AI and web technologies:

  • Google Gemini API — The core of VisiPillar's intelligence. We leveraged Gemini's powerful vision capabilities to analyze images of construction equipment and identify damage in real time
  • Frontend — A clean, mobile-friendly interface that allows workers to capture or upload images directly from the job site
  • Backend — A robust API layer that processes images, communicates with Gemini, and compiles structured damage reports
  • Report Generation — Automated digital report output with damage descriptions, timestamps, and severity classifications

Challenges we ran into

Building VisiPillar wasn't without its obstacles:

  • Prompt engineering — Getting Gemini to return consistent, structured, and actionable damage reports required significant experimentation and iteration
  • Variability in conditions — Construction sites are unpredictable. Accounting for poor lighting, muddy equipment, motion blur, and extreme weather pushed the limits of what the model could reliably interpret
  • Defining damage severity — Establishing a consistent, meaningful scale for categorizing damage (minor, moderate, severe) required careful thought and iteration
  • Balancing speed and accuracy — We needed responses fast enough for on-site use without sacrificing the accuracy that safety-critical reports demand

Accomplishments that we're proud of

  • Successfully built a working end-to-end AI pipeline — from image capture to structured damage report — within our development timeline
  • Leveraged the Gemini API to deliver impressively accurate damage analysis across a wide variety of equipment and conditions
  • Designed an interface simple enough for any worker to use, regardless of technical background
  • Created a tool that has real, tangible potential to save time, reduce costs, and improve safety on job sites around the world
  • Built something that genuinely respects and uplifts the people doing the hard work every day

What we learned

This project taught us far more than just technical skills:

  • Domain knowledge matters — Understanding how construction equipment is used, maintained, and inspected shaped every design decision we made
  • Prompt engineering is an art — Crafting precise, well-structured prompts was just as important as the code itself when working with large language models
  • Simplicity is hard — Building a powerful tool that feels effortless to use required countless iterations and user-focused thinking
  • Real problems deserve real solutions — Working on something with genuine human impact made every challenge worth pushing through

What's next for VisiPillar

VisiPillar is just getting started. On the roadmap:

  • Expanded equipment coverage — Support for a wider range of Caterpillar models and machinery types
  • Predictive maintenance insights — Using historical damage data to flag equipment at risk before failures occur
  • Fleet management integration — Syncing reports directly with Caterpillar's existing fleet management and maintenance platforms
  • Voice-to-report functionality — Allowing workers to verbally describe damage for hands-free reporting in difficult conditions
  • Multilingual support — Making VisiPillar accessible to construction crews around the world, regardless of language

The goal is to make VisiPillar the industry standard for equipment damage reporting — putting smarter tools in the hands of the workers who need them most.

Built With

Share this project:

Updates