Cosmos BIM Connect

Cosmos BIM Connect democratizes BIM data access with AI, enabling effortless querying in natural language for instant insights and enhanced collaboration.

Comment

## Inspiration

Unlocking data in the AEC (Architecture, Engineering, and Construction) industry can significantly address challenges like sustainability, energy consumption, and efficiency. The construction sector is responsible for approximately 39% of global carbon emissions, with 28% from operational emissions and 11% from building materials and construction activities.

By leveraging data, stakeholders can optimize designs, reduce waste, and enhance building performance. This data-driven approach fosters innovation and collaboration, enabling the industry to overcome its historical resistance to change and adopt more sustainable and efficient practices. Still, there is a significant gap in accessibility to crucial information embedded within Building Information Models (BIM). These models contain a wealth of data vital for project stakeholders, yet much of it remains inaccessible to those lacking specialized expertise in querying such complex datasets. Typically authored in 3D with tools demanding extensive training, BIM data can be formidable to navigate without technical proficiency. My vision was to bridge this gap by exporting BIM data into an external database, thereby enabling querying through a user-friendly language comprehensible to non-specialists. By employing vectorization techniques and leveraging Natural Language Processing (NLP) alongside LLM and RAG methodologies, I aimed to democratize access to this invaluable information. This innovation promised to empower diverse stakeholders with insights crucial for enhanced planning, accurate costing, and effective risk management, unlocking previously inaccessible opportunities for informed decision-making and project success.

## What it does

The software tool I create facilitates the extraction and selective migration of data from a BIM (Building Information Modeling) authoring tool. It intelligently stores this data in CosmosDB, utilizing OpenAI to generate embeddings that capture nuanced semantic relationships within the information. Using LangChain, the tool develops a sophisticated agent capable of accurate analysis and decision-making based on the dataset. Finally, deployed on Azure App Services, the tool provides a user-friendly interface for stakeholders to access and utilize its advanced functionalities seamlessly.

In essence, this software tool empowers users in the architecture, engineering, and construction industries to leverage extracted BIM data for enhanced decision-making, analysis, and operational efficiencies through advanced AI-driven capabilities.

## How we built it

Here’s how I orchestrate the process:

  1. Data Extraction and Migration from Revit: The journey begins with extracting crucial data from the BIM (Building Information Modelling) authoring tool. This step is not merely about extraction but also about selectively migrating data that is pertinent to the task at hand. It demands a deep understanding of the BIM authoring software to ensure that the extracted data retains its integrity and relevance. This data is then prepared to transition into a different database, setting the stage for subsequent operations.
  2. Storing Data in CosmosDB and Generating Embeddings: Once extracted, the next pivotal move is storing this curated data in CosmosDB. This choice is driven by the need for a scalable and globally distributed database service that seamlessly integrates with our workflow. Additionally, leveraging the power of OpenAI, we generate embeddings from the stored data. These embeddings serve as essential representations that encode the semantic meaning and relationships within the data, facilitating more nuanced analysis and processing down the line.
  3. Developing an Agent in LangChain: With data primed and embeddings in place, the focus shifts to creating an intelligent agent using LangChain. This platform not only enables the development of sophisticated language models but also provides a robust environment for training and testing our agent. The accuracy of our agent is meticulously validated against the dataset, ensuring that it meets stringent performance benchmarks and delivers reliable outcomes.
  4. Deployment on Azure App: Finally, our meticulously crafted software tool is ready for deployment. Azure App Services provide the ideal deployment environment, offering scalability, security, and reliability. This deployment phase marks the culmination of our efforts, making our tool accessible to users who can benefit from its capabilities seamlessly.

## Challenges we ran into

Extracting data from Revit models involved tedious manual selection and filtering of useful information. While exporting all model data was an option, it often included redundant nested objects or irrelevant details for our app prototype. To streamline this process, we focused on identifying and extracting a core set of parameters for testing, rather than attempting to handle the entire breadth of data available. This approach allowed us to optimize efficiency and relevance in data extraction for our specific application needs.

Learning new concepts and implementing them posed another significant challenge during the project. One example was applying the use of chains in LangChain, understanding how to extend their functionality to ensure accurate results based on user queries. This required a lot of experimentation to achieve desired outcomes.

Additionally, technical issues encountered in Phase 1 was slowing down progress. Using the Discord channel proved invaluable in troubleshooting and resolving these issues. I am grateful to everyone in the community for their openness and helpfulness.

## Accomplishments that we're proud of

Creating an App Integrating with GenAI for the AEC Industry despite not having prior knowledge of the main technologies involved is something I am proud of. This project challenged me to learn quickly and adapt, and apply new technologies effectively to real-world problems, learning and exploring a lot of solutions along the way.

Another achievement I take pride in is I have learned a lot of Azure, as it has been in my bucket list for a while. This will help me develop better apps and create better experiences in the cloud.

And finally, I am very proud that I managed to create a working app!

## What we learned

First off, MongoDB. It's been all about understanding how to handle data—creating, reading, updating, and deleting entries. It's like learning the language of modern databases.

Then there's the world of search. I've been exploring the difference between basic search methods and more advanced techniques like vector embedding. It's fascinating how these methods can find and relate information in smarter ways.

I also dabbled in Langchain's chains—a cool blend of language processing and blockchain tech. It's cutting-edge stuff, imagining secure applications that can understand human language seamlessly.

And finally, using ChatGPT for querying databases. It's been surprisingly handy, turning complex data tasks into simple conversations. It's like having a virtual assistant for data!

Each of these experiences has been eye-opening, showing me just how vast and exciting the AI world can be.

## What's next for Cosmos BIM Connect

Looking ahead, here are a couple of key improvements I envision for my app:

  1. Automatic Export of Data from Revit Models: One of the next steps I'm considering is implementing an automatic data export feature from Revit models. This could involve triggering a database export every time a model is saved. By automating this process, users can seamlessly update and synchronize data without manual intervention, enhancing efficiency and accuracy in data management.
  2. Integration with Web Visualization Apps like Speckle: Another exciting prospect is integrating my app with web visualization platforms such as Speckle. This integration would enable the visualization of results beyond simple text or schedules. Users would be able to interact with and explore data in a more dynamic and visual format, enhancing communication and understanding of project outcomes.

Built With

Share this project:

Updates