Foreman

AI app for Hispanic contractors who need back office support to keep organized and scheduled.

Comment

Inspiration My wife owns a Hispanic accounting firm, and every day we see contractors who know how to do the job but don't understand the back office side of the business. They know how to install siding or frame a house, but they don't know how to put together an invoice, how to keep track of their jobs, and they don't know what a mechanics lien is — so when someone doesn't pay, they don't know what their rights are. A lot of times they just accept it and move on because they don't know where to go to get help. That is not fair to them.

I built this app for them, so they can keep track of their jobs, who still owes them money, and the remedies they have if someone does not pay.

What it does Foreman turns a conversation into the back office.

The contractor speaks the job out loud in Spanish. The app cleans it up, translates it into English for the client. It turns the job description into a proposed contract that spells out the work — then the client signs it electronically.

The app organizes the materials, the timeline, and the crew. It tracks the budget against real spending — photograph a material receipt and Gemini reads the merchant and total and updates the job's margin.

It also documents the site itself. Before the job starts, the contractor logs "before" photos of the site condition. When the work is finished, he logs the "after" photos. Those become the proof attached to the final invoice — the app won't generate it without them.

This matters for two reasons. It shows the client the work was actually done, which is how a contractor gets paid without an argument. And it protects him from liability: a homeowner who claims the crew cracked a driveway or damaged a wall is answered by a timestamped photo of that driveway before anyone set foot on the property. Contractors get blamed for pre-existing damage all the time, and without a record it's his word against theirs.

Then it starts a Virginia mechanics-lien deadline on every unpaid job, counting down so the contractor knows exactly when he has to file to protect his money.

There is no language setting to get right. In the demo video, the language selector is deliberately left on English while I speak the job in Spanish — the audio goes straight to Gemini, which detects the language on its own and still returns the correct client, phone number, address, budget, and materials in English. The contractor speaks however he speaks. The paperwork comes out in the language the client needs.

How we built it Foreman is a Next.js and React front end with a Windows Metro-style tile interface, backed by Supabase (Postgres) for jobs, invoices, contracts, and documents. It's containerized with Docker and deployed on Google Cloud Run in us-east4.

The intelligence runs on Google's Gemini 3.7 Flash through the @google/genai SDK. The voice intake sends the recorded audio itself to Gemini as multimodal input rather than a machine-made transcript, so the model hears the contractor directly — that's what makes the language detection work. Gemini also handles the structured extraction, the Spanish-to-English translation, vision OCR on material receipts, and the morning briefing synthesis. A Google-hosted Gemma endpoint handles the lighter text tasks.

Model calls run through a cascade — Gemini 3.7 Flash, then 3.6, then 3.5 Flash-Lite — so a single model being unavailable doesn't take the whole agent down. The spoken briefing uses Google Cloud Text-to-Speech with Chirp 3 HD voices, and Google Maps Platform supplies supplier and drive-time context.

Every capability lives in its own isolated API route with its own failure handling, and all state lives in Postgres rather than in memory, so the agent is restart-safe.

Challenges we ran into The hardest part was the AI coding assistant I was building with. It would confidently report that a feature was "built and verified," and when I checked, the code hadn't changed and it had pasted fake passing test results. I had to build discipline around it: every change had to prove itself by actually running — real API calls, real outputs on screen, real rows in the database — not a summary claiming success. One three-word label change came back as 28,605 line changes across 30 files because it silently rewrote every line ending in the repo; I had to prove that with a diff and lock it down before it happened again.

Getting the AI to reason about mechanics-lien law took far more care than a "summarize this" prompt. Virginia's deadline is date-sensitive and unforgiving — 90 days from the last day of the month the work was done, with a 150-day lookback — so the math had to be exact and computed from real project dates, not estimated by a model.

Language was its own problem. The first version relied on the browser's speech recognizer, which has to be told the language in advance. A Spanish speaker on an English recognizer produced mangled phone numbers and addresses. I added a manual toggle, and it worked, but it put the burden back on the contractor. The real fix was to stop feeding the model a machine transcript and give it the audio itself.

Accomplishments that we're proud of I am really proud of the fact that a Spanish speaking contractor can input in Spanish and get the output in English. This helps out a lot because the contractor is busy working. Instead of having to hire people, he just speaks into the app and the app produces the invoice when he is done, emails and texts the clients an invoice with pictures and keeps track of unpaid invoices. The AI keeps track of the timeline for mechanics liens so the contractor can file one if necessary to guarantee payment. The AI is also a hybrid pipeline of online and offline. It works with or without a connection to the internet.

What we learned Mapping out my idea was easy. Getting the AI to build it, and build it correctly, was hard. To make sure it was actually building and not just telling me it was building, I had to verify every claim against what the code does.

I also learned that giving a model the raw signal beats giving it a cleaned-up version. Every layer I put between the contractor's voice and Gemini was a layer that could lose the phone number.

And I learned that my wife's clients don't have anything like this. They're curious about it and they want to try it. That's why I'm still building it.

What's next for Foreman Fully autonomous lien tracking. Today the morning briefing is prepared when the contractor asks for it. Next it runs on its own overnight, checks every unpaid job, and leads the briefing with any lien deadline that's coming up — without the contractor asking.

An evening briefing so they know what they got done that day, what's still unscanned, and what's still unpaid.

Email and SMS invoicing with job photos. Building the invoice document itself first, then sending it with before-and-after pictures attached, so there's a paper trail the client can't dispute.

A master command button — say "add labor to the expenses" or "pick up paint at the nearest hardware store" and the agent updates the right job or sets the reminder.

Offline capability. Contractors work in basements, new construction with no service, and rural sites. I want the app to keep taking voice notes and job updates with no signal and sync when it reconnects — with a small on-device model handling what it can in the meantime.

Other states' lien laws, so the same protection works outside Virginia.

My goal is to make sure the people who do the work get paid, and don't get taken advantage of — which happens a lot.

Built With

Share this project:

Updates

Submission history