Street Smart

Inspiration

On a large campus, there is uncertaintly whether you will run into an unfiltered water fountain or dirty bathroom. At Georgia Tech, there is no single source to learn which bathrooms are clean, water fountains are actually filtered, or which elevators are working, and not out of operation. Problems are known after the fact. That is, individuals will arrive at their resource after a long journey, only to find out that the elevator is out of service or the water fountain is not filtered. This is especially a bigger problem for individuals with mobility issues because a broken elevator could mean a longer detour or not being able to access a restroom if it is not working. We wanted to close the gap in accessibility issues on campus which inspired us to create Street Smart, an AI-powered campus map that shows which elevators, water fountains, and elevators are unsanitary or inoperational right now.

What it does

Street Smart is an AI-powered campus map that shows which elevators, water fountains, and elevators are unsanitary or inoperational right now. It works by users reporting issues and rating places, so that other users can see if there is a broken water filter or see how clean and accessible a bathroom is.

How we built it

Part 1 - The Pinpointing Map Built with React and Vite frameworks, Street Smart combines Google Maps' interactive interface with the convenience of the AI-enabled reporting tool. During the development stage, we brainstormed the interactive aspect of the map, providing examples of if a user can rank how good a resource is or how far one resrouce is away from the individual. Ultimately, we decided that the user can pinpoint a new resource on the map by providing an address and rank the resource at that place. For the pinpoint to work, we used Google's Geocoding API to turn the address into a set of coordinates.

Part 2 - The Reporting Agent Once the user identifies a problem with the resource, using the coordinates and the photo taken from the user, the integrated Gemini Flash model analyzes the user's problem with the resource. Then, it creates a report for that resource by filling out indicated subfields such as location and specific problem. After that report is submitted, the report is stored in our SupaBase platform to create a database of all reports, places inputted, and the rankings of the places. This database is the connection port to all the pinpoints visible on the map and reports gathered.

Challenges we ran into

As members new to using front end development and image-enabled functionality, there were several challenges:

1. Deciding how map details should be stored Since we wanted to answer the questions where a resource is exactly located and what exactly is wrong with the resource, it was first difficult to decide which methods to use. For example, how the reporting tool recognizes where the resource is, initially it was hardcoded but we changed the flow to take the loaction from the Geocoded coordinates in the frontend.

2. Backend to Frontend Communication We were using multiple API functionalities, so it was important for us to integrate them safely and accurately in our backend. Connecting them to the frontend posed new challenges as we had trouble determining where we would place the API keys. However, we discovered that creating an env file would solve the issue.

3. Unfamiliarity with Map API's Ensuring that the resources mapped and how an address can be converted to coordinates were learning curves for our team. Originally, the mapping was based off just one Google Javascript API, however we discovered that we needed the Geocoding API to unify the map placement for both the frontend and the backend.

Accomplishments that we're proud of

We are proud that our all-women hackathon team built a tool that will be usable by students immediately after HackGT. Not only did we build a full-stack, end-to-end pipeline during the short time we had, but we also targeted a prominent social issue on campus. We built an app that can drop a pin, snap a photo, have Gemini Flash interpret the issue and autofill a structured report, and make it visible to others on the map. Getting Geocoding, Google Maps, an LLM vision model, and a database to cooperate smoothly while our team was mostly new to frontend and image-enabled tooling was a big milestone for us. We are also proud that we have built something that addresses a real accessibility gap on our campus. It was rewarding to build something that could make Georgia Tech easier to navigate for students with mobility needs.

What we learned

Going into this hackathon, most of the team had not touched a real-world API before and integrated an API to this extent. We learned how geocoding turns a human-readable address into something plotable by a map, how to structure a multimodal AI pipeline, and how to keep a database in sync with a live map so that reports show up in real time. A non-technical lesson our team learned how to scope down our project in order to finish the project in time. At first, we wanted to have rankings, distance calculations, and more resource types.

What's next for Street Smart

Expanding to More Resrouces Currently, the only resources we have on our map are bathrooms, water fountains, and elevators. In the future, we would like to expand this to include other resources such as microwaves, printers, and trash cans.

More Test Cases Across Various Student Input With most of our team new to web development and APIs, nearly every step was a learning opportunity. We learned to keep our API key secure by routing requests through a Vercel serverless function, stored photos and reports in Supabase with permission rules that let students add reports but never delete them, and deployed the app so it works on any phone. By testing on our own phones and iterating on real feedback, we built a working, deployed prototype that turns a single sentence into an accurate, map-ready report.

Implement Across Different Campuses Street Smart is related to Georgia Tech's campus currerently. Our goal is to make our Gemini analysis model be implementable and scalable across different college campuses.