The Inspiration
The inspiration for this project came from a simple, yet profound realization: for millions of people, the freedom to navigate the world is filled with uncertainty. We watched a close coworker, who uses a wheelchair, describe the constant "scavenger hunt" she faces every time she leaves her home. A simple coffee meeting or a trip to a new part of town is preceded by a series of phone calls, hopeful internet searches, and often, the anxiety of arriving at a location only to find a flight of stairs at the entrance or a restroom that is inaccessible.
This daily struggle for basic information is a significant barrier to independence and spontaneity. The world is full of data, yet the crucial information needed for accessible navigation—real-time, reliable, and specific—is often scattered, outdated, or nonexistent. We were inspired to create a solution that would eliminate this uncertainty, empowering individuals with disabilities to move through the world with the same confidence and ease that so many of us take for granted. We wanted to build a tool that provides not just an answer, but a pathway to dignity and inclusion.
What We Learned
Throughout this project, the learning curve was steep but incredibly rewarding. Our biggest takeaway was a deeper understanding of the Americans with Disabilities Act (ADA) and its practical implications. We learned that "accessibility" is not a simple yes-or-no question. It’s a nuanced set of standards, from the width of a doorway and the height of a toilet to the presence of grab bars and the slope of a ramp. To provide truly useful information, the agent needed to understand and communicate these specific details.
We also learned about the power and complexity of data. Sourcing accurate accessibility information is a significant challenge. Public buildings might have available data, but private businesses are a different story. This project taught us the importance of data aggregation and the potential of crowdsourcing to fill in the gaps that official records leave behind. Finally, we gained a new appreciation for user-centered design. Building an AI agent that is itself accessible—with clear, concise language and simple commands—is just as important as the information it provides.
How We Built It
This AI agent was built on the Databricks Data Intelligence Platform, which was chosen for its ability to handle and process vast and varied datasets in real-time.
Data Ingestion and Processing: The foundation of the project is a multi-layered data lakehouse. We started by ingesting the nimble dataset on building permits and business listings. Using Databricks notebooks, we then integrated data from various APIs that provide location and accessibility information along with Open Street Maps data (OSM). All this data—structured and unstructured—is cleaned, unified, and stored in Delta Lake tables.
AI and Machine Learning: The core of the agent is the BAAI General Embedding (BGE) Large Language Model (LLM) fine-tuned on this specialized accessibility data. When a user asks a question like, "Is the coffee shop on Main Street accessible, and does it have an ADA entrance?" the model comprehends the natural language query. It then retrieves the relevant information from the Delta Lake tables, including the specific accessibility features on record.
Real-time Directions and Output: Once the agent confirms a location's accessibility status, it integrates with a mapping service API to provide turn-by-turn directions (didn't get this to work unfortunately). The final response delivered to the user is a comprehensive package: a clear confirmation of ADA accessibility, a list of specific features (e.g., "ramp at entrance, ADA restroom on ground floor"), and a direct link to directions.
The entire workflow, from data ingestion to the user-facing response, is managed and orchestrated within the Databricks platform, ensuring speed, reliability, and scalability.
Challenges Faced
We have a newfound respect for hostage negotiators after our experience trying to extract a single, error-free function from our AI assistant. There was a lot of back and forth, and at one point, we considered offering it a virtual pizza.
We spent 10% of our time coding and 90% in a high-stakes negotiation with our AI overlord for the right snippets.
Log in or sign up for Devpost to join the conversation.