ease 'n spect

Inspiration

Field excavator inspections are often a chaotic mess of inconsistent checklists and poor record-keeping. We built ease 'n spect to change that. It's a beautifully simple, AI-powered app that guides inspectors seamlessly, assesses equipment conditions objectively, and maintains crystal-clear records. The name plays on "inspect"—because we're making the entire process effortless and reliable. Through the simplified inspection process, we can perform inspections more frequently, increasing the probability of discovering critical defects.

What it does

ease 'n spect transforms complex heavy-machinery inspections into an elegant, step-by-step mobile experience.

Hitting Start Inspection launches an intuitive five-step flow (front cab window, left track, arm joint/hydraulic pump, and two bucket teeth views). Instead of guessing what to look at, users are guided by a sleek 3D visual cue that mimics the exact camera angle needed. They snap a photo, tap a condition (Good / Needs Attention / Bad), and effortlessly record a voice note.

Behind this simple UI is a powerful backend. Once uploaded, our multimodal AI (running on Modal's infrastructure) takes over: Gemini performs vision analysis for operational fitness and wear, Whisper transcribes voice notes, and Qwen 3 suggests replacement parts. The final output is a clean, scannable summary card that clearly separates the user's input from the AI's assessment, complete with intuitive visual wear indicators and easy-to-read recommended next-steps.

How we built it

Frontend: React Native + Expo (Router, Camera, AV, Haptics) for a fluid, tactile mobile feel. We integrated React Three Fiber + drei to render the interactive 3D excavator and teeth models that seamlessly guide the user's camera.

Backend: FastAPI paired with Supabase (PostgreSQL). We utilized Gemini (gemini-2.5-flash) for vision analyses—validating and parsing responses smoothly with Pydantic—in conjunction with open source models such as Whisper (STT) and Qwen 3 (parts suggestion) pipelines.

Challenges we ran into

Dynamic 3D Viewport Orchestration: We wanted the app to visually show, not just tell. Mapping our React Three Fiber excavator model to specific, dynamic camera viewports for each of the five inspection steps was mathematically and technically demanding. The challenge was ensuring these spatial transitions felt fluid and elegant on a mobile device. We had to perfectly mirror the physical world to instantly cue the user on exactly where to point their camera, completely eliminating guesswork without introducing any clunky UI rendering lag.

Blueprinting an Invisible, Efficient AI Pipeline: Beneath our minimalist UI lies an intricate, heavy-duty web of AI models (Gemini, Whisper, Qwen) hosted on Modal. The challenge wasn't just connecting them, but blueprinting a pipeline that processes complex multimodal data (images, voice, text) efficiently. We had to meticulously architect our FastAPI backend—leveraging tools like asyncio.to_thread to offload blocking tasks and execute validations flawlessly. This architectural heavy lifting was essential so the UI remains incredibly snappy and responsive, never leaving the user waiting on a loading screen while massive amounts of data are crunched.

Accomplishments that we're proud of

Frictionless UX: A truly end-to-end flow where taking photos, logging conditions, and getting an AI breakdown feels completely natural and effortless.

Intuitive 3D Guidance: Aligning the 3D model's camera with each physical checkpoint so users immediately know what to photograph, removing all guesswork from the field.

Elegant Data Presentation: A polished UI that clearly separates "your condition" from "AI assessment," reusing a single, responsive summary view for both live runs and historical data.

What we learned

Data dictates Design: Schema-first AI (prompts + Pydantic) is crucial. Predictable, strictly parsed data makes for a highly stable, crash-free interface.

Less is More: Consolidating multiple images into one AI prompt results in a much cleaner, less redundant user experience.

Responsive APIs: Using asyncio.to_thread for blocking Gemini calls ensures the FastAPI backend—and consequently the UI—never hangs or feels sluggish.

What's next for ease 'n spect

Expanded Fleet: Bringing this intuitive flow to skid steers, loaders, and other machinery with their own tailored 3D models and prompts.

Fleet Dashboards: Beautiful, at-a-glance views to see what needs attention across an entire job site.

Offline Mode: Seamless caching so the elegant UX doesn't drop when the cell signal does.

Actionable Alerts & Ordering: Turning the "parts to replace" list into direct, one-tap catalog orders.

Built With

• expo.io
• fastapi
• modal
• react-native
• supabase
• typescript
• whispir