Building Brain

Building brains for buildings

Comment

Inspiration

Ever worked in a massive building and thought, "Hey, it's almost like this place has a mind of its own?" We did. As a team, we've spent a good chunk of our lives navigating the wild world of facilities management. In an industry that's traditionally lagging in tech, we noticed that facility folks are often under-equipped when managing large, complex buildings. Plus, managing a gazillion data sources like asset lists, manuals, and paper-based records can be a total headache. We knew we could do better, so we created Building Brain!

What it Does

Building Brain isn’t your run-of-the-mill management tool. Think of it as a one-stop-shop for all things building management. It stitches together your building's layout, asset info, telemetry-based diagnostics, and lets you have a legit chat with your building. Yes, you read that right! Got a problem like temperamental temperatures on a specific floor? Building Brain uses NLP to understand the issue, scans all the relevant equipment, and hooks you up with a step-by-step guide, straight from the manufacturer's manual, to diagnose and troubleshoot the problem. No more guesswork!

How We Built It

Building Brain is a mashup of all sorts of tech goodness. We started with a massive dataset from an actual building – asset info, manuals, the works. Then, we ran this data through a Language Model to add a layer of human-friendly context about each piece of equipment. This souped-up data was then vectorized and stored in Pinecone.

We connected that to another Pinecone index packed with manufacturer manuals for all the common systems in the building. We then took LangChain and Claude 2 and whipped up an API for submitting queries. The front-end user interface? All React, baby! So yeah, our tech stack is a healthy blend of React, MUI, PineconeDB, Langchain, Dynamo DB, and S3 for embedded file storage, plus Auth0 for login and user management.

Challenges We Ran Into

Venturing into this project, we ran into quite a few challenges. Primarily, our language model presented some hurdles. Initially, it had a tendency to yield inconsistent results. We used LangChain to train the model to determine what data it needed to retrieve based on the complexity of the user's query. For instance, LangChain helped differentiate between a "simple fix", for which an easy step-by-step solution would suffice, and a more complex issue requiring the retrieval of in-depth technical manuals and documentation.

However, achieving this level of precision and reliability required substantial fine-tuning and prompt engineering. It was critical for us to ensure that users always receive accurate responses, regardless of the complexity of their queries. As a part of this process, we used LangChain again to format the output. Depending on the need, the system was designed to handle pure JSON data or boolean operators for some logic flows.

A key part of our strategy was adding the context from LangChain as the first 50K-60K of the context window. This ensured that the model always had the most relevant, contextually appropriate information at its disposal. Despite these thoughtful measures, the model occasionally faltered in its precision, requiring us to dig even deeper into the nuances of prompt engineering.

On top of the language model complexities, managing the AWS setup and credentials turned out to be a much more time-consuming task than we had anticipated, adding another layer of complexity to the project.

Accomplishments That We're Proud Of

Despite these obstacles, we're really excited about what we've accomplished. Building Brain doesn't just iterate on the concept of "Smart Buildings," it evolves it. We genuinely believe this tool can make a tangible difference in the lives of facility managers, especially in an industry that's traditionally been slow to adopt change.

What We Learned

This project was a treasure trove of learning opportunities. We honed our skills in prompt engineering, unraveled the intricacies of language models, and mastered the efficient management of cloud resources. It was a phenomenal experience to stretch our technical capabilities and create something with potential real-world impact.

What's Next for Building Brain

We're just getting started! We're looking to integrate more awesome info sources like YouTube troubleshooting videos and historical maintenance records. We're also stoked about bringing CLIP supported image-based diagnostics and real-time telemetry data from IoT devices to the mix. This will turbocharge the model's ability to predict maintenance needs and manage equipment breakdowns.

Plus, we're super excited about hooking up Building Brain with the Yelp Fusion API. Why? Because it'll suggest the best local vendors, and that means a new revenue stream through lead generation. We're talking about seriously upping our Total Addressable Market (TAM). Hold onto your hats, because Building Brain is about to take the facilities management industry by storm!

Built With

Share this project:

Updates