PocketBall - Real Time Voice MLB helper

MLB - Pocketball

Inspiration

I was inspired by the OpenAI voice chatbot and its ability to engage in natural, interactive conversations. I wanted to create something similar but tailored specifically to the world of baseball—an app that could not only provide match insights but also chat like a friend, bringing MLB fans closer to the game.

What it does

PocketBall is a real-time voice-enabled MLB chatbot that enhances how baseball fans experience the game. It provides live game updates, including real-time scores, inning progress, and key plays. Powered by the Gemini Flash 1.5 model, PocketBall offers AI-driven game insights and strategic analysis during live matches, helping fans understand critical moments. The app also delivers comprehensive game summaries with time-stamped highlights for every major event, ensuring fans never miss the action. Additionally, users can access up-to-date player and team statistics. With Google Cloud’s Speech-to-Text and Text-to-Speech technologies, PocketBall allows natural, conversational interactions, making it as simple as talking to a friend. Its clean and intuitive interface ensures effortless navigation, whether checking live updates or accessing game summaries.

How we built it

Voice Recognition and Response: Implemented using Google Speech-to-Text for recognizing user commands and Text-to-Speech for generating lifelike voice responses.
AI-Powered Insights: Leveraged the Gemini Flash 1.5 model on Vertex AI to process and deliver real-time match analysis, strategic insights, and post-game summaries.
User Interface: Designed a clean and intuitive interface that allows users to quickly access features with just a tap.
Core Features: Built functionalities for live game updates, player stats, and time-stamped game event reviews to provide a comprehensive experience for baseball fans.

Challenges we ran into

The biggest problem we encountered was how to use the provided dataset API to deliver game data in real time and how to use the 2024 MLB Regular Season Schedule as Gemini's RAG to provide accurate answers. To this end, I created a script specifically for obtaining the "liveData-plays-currentPlays" information in the API, and set it to update it every ten seconds, save it as a json file, and then send it back to Gemini, so that Gemini can obtain the latest game data and provide users with game analysis.

Accomplishments that we're proud of

One of the biggest accomplishments we’re proud of is successfully integrating advanced AI technologies like Google Cloud’s various technologies, and the Gemini Flash 1.5 model into a seamless user experience. It is our first time experience of building a fully functional real time voice chatbot. We’re thrilled to have created a chatbot that delivers accurate, real-time MLB game insights and summaries, making it both functional and enjoyable for baseball fans.

What we learned

Through this project, we learned the importance of combining cutting-edge AI technologies with user-focused design to create a seamless experience. Working with Google Cloud’s Speech-to-Text technologies taught us how to enable natural and accurate voice interactions. We also gained valuable experience fine-tuning the Gemini Flash 1.5 model on Vertex AI to handle real-time baseball-specific queries and provide meaningful insights.

What's next for PocketBall - Real Time Voice MLB helper

In the future, PocketBall will integrate more advanced voice interaction capabilities, such as real-time interruptions and support for multiple languages, to make conversations even more dynamic and inclusive. Additionally, we plan to enhance the process of retrieving the latest MLB game updates, ensuring that Gemini can access and deliver information with even greater speed and accuracy.