ProcureMind AI — Project Story

Inspiration

The Procurement track brief put it plainly: construction procurement runs on contracts, purchase orders, invoices, and vendor relationships — and almost all of it still depends on manual review and disconnected systems. That framing matched something we kept noticing as we read into how construction sites actually operate: a truck can arrive carrying lakhs of rupees' worth of steel, and the decision to accept it or reject it still comes down to someone manually flipping between a purchase order, a handwritten delivery challan, and a contract.

What struck us wasn't that mismatches happen — mismatches are inevitable on any large project. It's that existing procurement software (SAP, Oracle, Procore, Autodesk Construction Cloud) stops at "there's a mismatch" and leaves the hard part — what does this actually mean for the project, and what should we do about it — entirely to a human, under time pressure, with the concrete already being poured. We wanted to build something that picks up exactly where those tools stop.

What it does

ProcureMind AI is an explainable procurement decision engine for construction. It doesn't just detect a mismatch between a purchase order, a delivery challan, and an invoice — it reasons about what that mismatch means for the project, and recommends the best response, with a human always approving the final action.

Concretely: a site engineer photographs a delivery challan — handwritten, crumpled, exactly as it looks on a real site — along with the delivered material. ProcureMind reads it with a vision-language model, checks the photo for visible material and damage inconsistencies, and lets the engineer confirm the exact quantity (we don't ask the AI to count steel beams in a photo — that's still a human call). From there, everything is deterministic: three-way matching against the PO and invoice, a contract clause check, and a Critical Path Method lookup against the project schedule to see whether the shortfall lands on a task with zero float. If 500 steel beams were ordered and only 470 arrive, and those beams feed a zero-float foundation task, ProcureMind doesn't just flag "quantity mismatch" — it estimates the schedule impact, lays out the real options side by side (wait five days for free, or use an approved backup vendor for ₹1.8 lakh with zero delay), and shows exactly why one option wins. The recommendation is drafted, not executed — a Project Manager reviews the comparison and hits Approve before anything is sent.

How we built it

We split the system into four stages that map directly to our architecture: Document Intelligence (VLM-based extraction with per-field confidence scoring, built specifically for messy, handwritten, real-world documents rather than clean scans), Procurement Intelligence (deterministic three-way matching plus RAG retrieval over contract clauses and vendor data), Construction Intelligence (a CPM lookup against a sample project schedule, feeding a transparent, configurable weighted risk-scoring formula), and Human Approval (a strict approval gate before any action executes).

The stack: React and Tailwind for the dashboard, FastAPI for the backend, PostgreSQL for storage, Gemini/Claude Vision for document and photo extraction, LangChain to orchestrate the pipeline steps, and ChromaDB for the RAG layer over contracts and vendor history. The one architectural decision we kept coming back to was keeping the LLM out of arithmetic entirely — matching, the CPM lookup, and the risk score are all plain deterministic computation; the LLM's only job is to synthesize those computed facts into an explainable, human-readable recommendation.

Since we don't have access to a real construction company's procurement records, we built a synthetic but realistically formatted dataset — POs, invoices, and delivery challans mirroring real Indian construction document formats, deliberately including some low-quality scans and injected mismatches — plus a small sample project schedule with float values, so the CPM lookup had something real to reason over.

Challenges we ran into

The hardest design decision wasn't technical, it was disciplinary: deciding what the AI should not do. It was tempting to let the vision model estimate quantities directly from a photo, but counting objects reliably from a crumpled, angled site photo is genuinely unreliable — so we drew a hard line: the AI verifies material type and visible damage, a human confirms the count. The same discipline applied to the risk score — early drafts of the formula implicitly asked the LLM to weigh cost against schedule impact, which meant two runs on the same input could disagree. Moving that into a fixed, configurable formula and restricting the LLM to explanation-only fixed it, but took a few iterations to get right.

Building a believable CPM lookup without a real project's schedule data was also harder than expected — we had to construct a small sample schedule with genuinely correct float values so that "zero-float task" wasn't just a label but an accurate derivation from task dependencies. And as a three-person, largely self-taught team, simply scoping the MVP honestly — resisting the pull toward the more ambitious six-agent, fully autonomous version of this idea — took real discipline. We kept the version that could genuinely be demoed end to end.

Accomplishments that we're proud of

We're proud that ProcureMind doesn't read as an LLM wrapper around a chatbot — every number a Project Manager sees traces back to a deterministic calculation or a retrieved document, and the LLM's role is confined to exactly the part it's good at: turning several computed facts into a clear, explainable recommendation. We're also proud of building the confidence-scoring and human-approval gates in from the start rather than as an afterthought — a system that pauses to ask "are you sure?" on a low-confidence field is a lot more trustworthy than one that silently guesses. And for a beginner team, getting a full pipeline — messy document in, explainable recommendation with a comparison table out — working end to end on synthetic data is something we're genuinely proud of.

What we learned

We learned a surprising amount of real construction scheduling theory — what "float" actually means, why a delay on a non-critical task barely matters while the same delay on a critical-path task cascades through an entire project. We learned, from trying it the naive way first, why enterprise AI systems separate deterministic computation from LLM reasoning, and why judges and real users alike trust a system more when they can see exactly which parts are calculated and which parts are generated. And we learned that scoping honestly — being explicit about what our MVP does today versus what's on the roadmap — made the project stronger, not weaker, because it meant every claim in our pitch was one we could actually back up.

What's next for ProcureMind

The next steps split into two directions. On the construction side: extending beyond POs, invoices, and delivery challans to reason over Bills of Quantities, material grade specifications, Goods Receipt Notes, and inspection reports, so the system's understanding of a project isn't limited to procurement documents alone. On the system side: generalizing the CPM lookup from our sample schedule to any project's real schedule data, moving from three-way to full five-way matching, expanding the single pipeline into genuinely independent agents with their own tool access, integrating with live ERP systems, and — once real historical delivery data exists — moving from our transparent scoring model to true probabilistic forecasting of vendor delays and price trends. Longer term, we'd like to pilot ProcureMind with an actual EPC contractor on a live project, where the real test isn't a synthetic dataset but a real truck at a real gate.