ForgeMind

For a detailed proposal: ForgeMind

Inspiration

Construction is one of the world's largest industries, yet the people making the most critical decisions on-site often have the least access to real-time project information.

While engineers work with BIM models, updated drawings, schedules, procurement systems, and project management platforms, workers on the ground still rely on verbal communication and fragmented updates. As a result, design revisions are missed, trades interfere, materials arrive out of sequence, and costly rework becomes inevitable.

We realized that construction does not primarily suffer from an information problem—it suffers from a coordination problem.

This inspired us to build ForgeMind, an AI Site Engineer that delivers the collective intelligence of the entire project directly to the people performing the work.

What it does

ForgeMind is an AI-powered construction execution platform built for the Physical AI track.

At its core is the Project Brain—a continuously updated digital representation of everything happening across a construction site. It combines live observations from wearable smart glasses with project drawings, BIM models, schedules, specifications, material availability, and historical knowledge to understand the current state of work.

Unlike traditional construction software that stores information or documents project progress, ForgeMind continuously reasons over project context to provide proactive recommendations before mistakes occur.

Workers interact with ForgeMind through smart glasses, receiving contextual voice guidance while keeping both hands free. Supervisors receive real-time alerts whenever coordination conflicts, safety risks, or installation issues are detected.

Our core innovation is a predictive recommendation engine that continuously learns from every intervention, mistake, and project outcome. Every completed project improves future recommendations, creating a continuously evolving organizational knowledge system.

How we built it

ForgeMind combines multiple AI techniques into a single reasoning pipeline.

Vision-Language Models (VLMs) understand what workers are seeing in real time.
Retrieval-Augmented Generation (RAG) retrieves relevant drawings, specifications, manuals, and project documents.
A Knowledge Graph models relationships between workers, locations, tasks, equipment, materials, schedules, and project dependencies.
The Project Brain continuously integrates live site observations with project knowledge to maintain an up-to-date understanding of construction progress.
A rule-based reasoning engine validates recommendations against engineering rules and project constraints before presenting them to users.
Confidence scores and explainable reasoning accompany every recommendation, ensuring engineers remain in control of final decisions.

Rather than functioning as another chatbot, ForgeMind acts as an intelligent execution layer that reasons continuously over the evolving state of a construction project.

Challenges we faced

One of our biggest challenges was defining a solution that went beyond simply attaching an LLM to a pair of smart glasses.

Early in our design process, we explored several directions including safety monitoring, installation verification, and voice assistants. While these ideas solved individual problems, they did not fundamentally improve how construction projects are executed.

Through multiple iterations, we shifted our focus toward the broader challenge of coordination and decision-making. This led us to develop the Project Brain, enabling ForgeMind to reason across project data sources rather than reacting to isolated events.

Another challenge was balancing automation with trust. Construction is a safety-critical industry where AI recommendations must remain transparent and explainable. To address this, ForgeMind always provides evidence for its recommendations and keeps engineers responsible for final decisions.

Our design philosophy became simple:

ForgeMind recommends. Engineers decide.

What we learned

Building ForgeMind taught us that successful AI systems are not necessarily those that automate the most work—they are the ones that deliver the right information to the right person at the right moment.

We also learned that the biggest opportunity in construction AI lies not only in perception, but in reasoning. Cameras can detect objects and documents can store information, but coordinating workers, schedules, materials, and design changes requires contextual understanding across the entire project.

Finally, we realized that every project generates valuable knowledge that is often lost once construction finishes. By continuously learning from project outcomes, AI can transform temporary project experience into permanent organizational intelligence.

Future Roadmap

Our initial deployment focuses on high-value construction activities such as electrical and MEP installations, where sequencing errors often lead to expensive rework.

Future versions of ForgeMind will expand across multiple trades, integrate more deeply with enterprise construction platforms, and build organization-wide learning systems that continuously improve execution quality across every new project.

Our long-term vision is to make every construction worker as informed as the project manager—without requiring them to stop working or search through documentation.

Built for Kaya AI

Kaya AI's vision centers around Amber, an AI agent that understands project context across construction workflows.

ForgeMind extends this vision to the point of execution.

Amber understands the project.

ForgeMind delivers that intelligence to the worker exactly when a decision matters.

Together, they create a continuous loop between planning, execution, learning, and future project improvement.

Vision

Buildings are rarely delayed because drawings are missing.

They are delayed because knowledge reaches the wrong people too late.

ForgeMind ensures that every worker builds with the collective intelligence of the entire project—and that every completed project makes the next one smarter.