BuildMate — AI Procurement for Home Builders

Inspiration

Construction is one of the largest industries on earth and one of the least served by good software. We watched a family member try to build a home and lose lakhs to a problem nobody talks about: procurement chaos. They had no idea how much cement or steel they actually needed, no way to know if a vendor's price was fair, and no visibility into deliveries. Studies suggest up to 30% of construction materials are wasted, over-ordered, or unaccounted for and individual builders and small contractors have zero tools to fight it.

We also realised the same blind spot exists in public construction: government bodies hand contractors large lump-sum budgets with little independent verification of what a project should cost. We asked a simple question: what if you could upload a house plan and have an AI handle and audit the entire buying process for you?

What it does

BuildMate turns a house plan into a finished procurement workflow in minutes:

Reads your plan — Upload a floor plan image and a Vision-Language Model (Qwen2.5-VL) extracts the built-up area, room count, walls, and openings as structured JSON.
Builds your shopping list — A RAG-grounded construction formula engine converts that geometry into a full Bill of Quantities (cement, steel, bricks, sand, tiles, paint) with every number editable by the user.
Forecasts price & risk — A forecasting layer predicts material price trends ("cement rises ~8% in monsoon, order now") and per-vendor delivery-delay risk, generating the signals the optimizer needs.
Optimizes procurement — Instead of picking the cheapest item, a linear-programming solver finds the cheapest combination of vendors across the whole order, factoring delivery fees, minimum order quantities, and lead times. It shows exactly how much you save versus a naive order.
Negotiates for you — An AI agent auto-drafts and sends emails requesting quotes, then reads vendor replies, extracts quoted prices, reconciles their quotation PDFs against the BOQ, and counter-offers or confirms.
Tracks delivery — You get status updates and notifications from order to doorstep.

How we built it

Frontend: React + Tailwind for a clean, demo-ready interface.
Backend: Python + FastAPI orchestrating the pipeline.
Plan reading (VLM): Qwen2.5-VL, an open-weight Vision-Language Model, reads floor plan images natively and returns structured JSON — no custom CV model to train. Optional zero-shot detection (YOLO-World) draws bounding boxes on rooms and openings for a live visual.
Materials estimator (RAG): A hybrid approach, the VLM extracts geometry, then deterministic quantity-surveying formulas, retrieved from a store of construction standards, convert it into quantities. The AI reads; the math calculates. This kept us accurate instead of hallucinating numbers.
Forecasting: A lightweight time-series / probabilistic model estimates material price trends and per-vendor delivery-delay risk from historical data, feeding those signals into the optimizer.
Procurement optimizer: A linear-programming solver (PuLP) minimizes total cost across the full order, choosing the cheapest combination of vendors under real constraints (delivery fees, minimum order quantities, lead times), not just the cheapest item.
Negotiation agent (Agentic): A multi-step LLM agent that drafts context-aware emails, sends them via the Gmail API, reads vendor replies, extracts quotes via document intelligence, reconciles quotation PDFs against the BOQ, and counter-offers or confirms.

Technique stack: the system deliberately combines five AI families, VLM (plan reading), RAG (grounded estimation), Forecasting (price/risk), Optimization (vendor selection), and Agentic + Document Intelligence (negotiation and reconciliation) rather than wrapping a single prompt. A core formula in our estimator, for example, estimates cement as cement bags = built-up area (sq ft) × 0.4, with similar rules for steel, bricks, sand, aggregate, tiles, and paint, stored in a config file so we can add regional presets later.

Challenges we ran into

Making the AI trustworthy, not just clever. Asking an LLM "how much cement do I need" produces confident, wrong answers. Splitting the job, VLM for reading, formulas for math, was the breakthrough that made our output reliable.
Human-in-the-loop. We turned "the AI might misread the plan" from a weakness into a feature by letting users edit every extracted value before ordering.
Scoping for a hackathon. We resisted building a full vendor-onboarding marketplace and instead focused all our energy on the AI pipeline, seeding realistic vendor data so the intelligence could shine.
Procurement is a real optimization problem. "Cheapest per item" is often wrong once delivery fees, minimum order quantities, and lead times enter the picture. Framing vendor selection as a linear program rather than a sort was the difference between a matcher and a genuine optimizer.

Accomplishments that we're proud of

A working end-to-end pipeline in a hackathon window — from plan image to optimized order to agent negotiation that actually runs, not just mockups.
We made AI estimates trustworthy. By splitting reading (VLM) from calculation (formulas), our Bill of Quantities is grounded and defensible rather than hallucinated.
We turned vendor selection into a genuine optimizer that produces a concrete, quantifiable saving versus a naive single-vendor order, a number a real buyer would care about.
An agent that does, not just drafts — it reads vendor replies, extracts quotes, and reconciles them against the order, closing the loop end-to-end.
A public-good angle: the same engine can independently audit public-works budgets, giving the project impact well beyond individual home builders.

What we learned

Domain knowledge beats raw model power. Construction thumb-rules made our estimates credible in a way no prompt could, the right formula in a config file outperformed a clever prompt.
The best AI products do things, they don't just answer. Our agentic email flow : sending, reading, reconciling; is what makes BuildMate feel like a real assistant instead of a chatbot.
Combining techniques beats scaling one. Layering VLM, RAG, forecasting, optimization, and agentic document intelligence created something none of those pieces could do alone.
Build the safe path first. Shipping an end-to-end pipeline with typed input before adding the risky vision layer meant we always had a working product to demo, whatever went wrong.

What's next for BuildMate

Fighting corruption in public construction. Government bodies currently release large lump-sum budgets to contractors with no independent check on what a project should truly cost. BuildMate can generate a standards-grounded, line-by-line cost estimate for any public project before funds are released, exposing inflated bids and padded quantities, and giving auditors a transparent baseline. Applied to public tenders, this turns procurement from a black box into an accountable, verifiable process.
CAD/BIM file support for professional builders and architects.
Regional cost presets (coastal vs plains, RCC vs load-bearing, multi-storey).
Real logistics integration for live delivery tracking.
A subcontractor and labour-cost layer to estimate the full project budget, not just materials.