Buddy

Buddy is your personal voice-powered app designed to find Bus transit routes, AI powered itinerary summaries and Photorealistic 3d Map animation for the selected Bus Route.

Comment

Inspiration

Over 3 million students rely on Shuttle-UM buses at the University of Maryland each year. While the Transit App offers reliable real-time tracking, I noticed a gap: it felt too cluttered. Buddy makes navigating campus buses and Metrobus as simple as saying, “Find me a bus to Regents Drive Garage?”

What it Does?

  • Listen to natural speech
  • Understand bus-related queries like:
    • "Find me a bus to Eppley Recreation Center.”
    • "Find me a bus from Graduate Hills to Regents Drive Garage.”
  • “Show me the bus route”:
    This would create a photorealistic 3D map animation for all the bus stops.
  • "Where am I right now?":
    Automatically fetch your location using GPS coordinates and display it.
  • Retrieve and summarize the bus transit itineraries.

How its built ?

  • Flutter + Dart: Built the mobile interface for speech interaction, message rendering, etc.
  • Speech-to-Text: Used speech_to_text package to convert voice to text in real-time.
  • Flask Backend: Acts as a middleware between the app and third-party APIs (Dialogflow, TransitApp, Maps, Gemini).
  • Dialogflow: Handles NLP for understanding intents.
  • Geolocation: Fetches user coordinates and place names using the Geolocator package and Google Maps Geocoding API.
  • Photorealistic 3D Tiles API: Used to show animation and bus stops.
  • Transit Routing: Calls TransitApp’s OTP planning API for real-time itinerary results.
  • Google Gemini AI: Parses JSON route data and generates clean, human-friendly summaries with coordinates and dynamic Google Maps links.

Challenges:

  • Geolocation accuracy: Translating location intents to real GPS data was tricky, especially with ambiguous addresses.
  • 3D Maps: Figuring out the animation with each bus stop for the selected bus itinerary was difficult because this feature is in ALPHA by Google.
  • Processing .json file having bus routes.

Accomplishments that I'm proud of:

  • Generated interactive route summaries with Gemini AI that users can explore via photorealistic 3D maps.

What I learned

Combining Flutter , flask and a bunch of API’s and create a fully working prototype

What's next for Buddy?

More extensive summaries for Bus routes with Gemini API Add real-time bus tracking. Lot of error handling needed for edge cases. Integrate Database and multi-users.

Built With

Share this project:

Updates