Vision Service Demo
Dashboard
Sunny View
Waiting on Ride
Scheduling a Ride

Inspiration

Autonomous Ride sharing is becoming a prevalent form of transportation. It's convenient and exciting, but it still falls short, especially in human reasoning. It lacks the human judgment and context awareness a driver brings. A human driver might pull up closer to a door when it's raining, or pick a flatter spot for someone with a walker. Autonomous vehicles don't make those small adjustments yet, and those adjustments make a big difference in the rider experience.

What it does

We built a ride sharing feature for autonomous vehicles that will allow for passengers to be picked up and dropped off in accessible areas. The feature will take into account context such as weather and the user's physical ability. It works in two scopes. Predictively, when a ride is requested, it uses the rider's accessibility profile, weather forecasts, and map data to select the best pickup and dropoff points ahead of time. In real time, as the vehicle approaches, it uses computer vision and live conditions to catch obstacles like blocked curbs or sudden rain and adjusts the spot quickly. Using this context, our feature will predict and choose in real time the best pick-up point that optimizes rider comfort.

How we built it

Before writing any code, we spent a significant chunk of time designing the architecture, because we knew we wanted multiple people building in parallel without stepping on each other. That's what pushed us toward a microservice architecture: each service owns one responsibility, exposes a clearly defined API contract, and can be developed, tested, and deployed independently. We used a swarm of agents to make our workflow more efficient, but we utilized human in the loop reviewing and pair programming to validate the agents code.

Challenges we ran into

Publicly available data on curbs and accessibility wasn't complete enough for what we needed, so we pulled from a variety of sources to assemble the datapoints our model relies on. We also struggled with narrowing our scope. There are many directions this project could go, and we had to prioritize what was achievable within the hackathon's time and resources. We hope to enhance this project in the future.

Accomplishments that we're proud of

We're proud of building a working system that turns messy, incomplete real-world data to give context for real-time pickup decisions. We also learned how to plan and divide up a project so everyone could build in parallel and bring it together smoothly. Most of all, we're proud that we took an ambitious idea, learned to scope it down to what really mattered, and still shipped a working feature.

What we learned

We learned how much accessibility data is missing or fragmented in public datasets, and how important it is to validate data from multiple sources. We also learned how to structure a project so that AI agents can contribute effectively, and why human oversight is still essential for catching mistakes and keeping the architecture coherent.

What's next for endpoint

Our next step is conducting user studies with riders who have disabilities to understand what they actually want from a feature like this, rather than relying on our own assumptions. We also plan to expand the data Endpoint takes in, adding more sources and context so its pickup and dropoff decisions become more accurate and useful in more places.

On the technical side, we plan to replace our reliance on a general-purpose LLM for the computer vision component with specialized models trained or tuned on accessibility-focused imagery. Purpose-built detectors for features like curb cuts, ramps, stairs, and sidewalk obstructions would be faster and more accurate than prompting a general model, and would make real-time inference on vehicle hardware more realistic.

We also want to expand the data pipeline behind Endpoint by integrating additional sources such as municipal infrastructure datasets, real shade/solar data and traffic feeds. Broader, higher-quality inputs will make its location scoring more reliable and let it generalize to cities beyond the ones with preexisting data sets.