GabbleGenie

• 

  Logo

• 

  Cover

• 

  Motivation

• 

  What we did

• 

  Potential impact

• 

  Next steps

Inspiration

We try to create an intuitive Gemini Pro 1.5 companion for anyone who feels overwhelmed by social events, providing the support needed to thrive in conversations.

What it does

GabbleGenie utilizes the Gemini API to analyze transcribed text conversations, including prior input user information, the conversation's environment, and conversational goals. It continuously updates the likelihood of achieving the goal and the potential awkwardness level for each branch and node in the decision tree of the conversation flow. This enables GabbleGenie to provide timely suggestions for conversation topics, helping users steer the discussion back toward achieving their goals when they are at risk of deviating. By offering guidance on initiating new small talk dialogues, GabbleGenie empowers users to navigate conversations effectively, even in situations where the original goal may be at risk.

How we built it

Code Implementation

GabbleGenie is built using Python for the backend logic and integration with the Gemini API. The codebase utilizes various libraries for building and analyzing conversation flow graph, and generating visualizations of the decision trees, and speech_recognition for transcribing WAV to text. The logic involves parsing transcribed text conversations, extracting relevant information such as user profiles, conversation environment, and goals, and using this data to update the likelihood of achieving the conversation goal and the potential awkwardness level for each conversation branch and node.

User Interface Design

The user interface of GabbleGenie is designed to be intuitive and user-friendly, with a focus on providing actionable insights and recommendations to the user. The interface features a clean and minimalist design, with easy-to-understand visualizations of conversation flow and decision trees.

Key elements of the user interface include:

• Phone Dashboard: A centralized hub where users can view and manage their ongoing conversations, access analytics and insights, and customize their preferences.
• AppleWatch Screen: The suggested expression is also shown on users’ watch in a subtle way during daily conversation.
• Conversation Analysis: Detailed breakdowns of conversation flow, including visualizations of decision trees with probabilities and awkwardness levels for each branch and node.
• Recommendations: Personalized suggestions for conversation topics, strategies for steering the conversation toward the desired goal, and tips for improving communication skills.
• Real-time Assistance: Interactive prompts and notifications to guide the user during live conversations, providing instant feedback and support.

Challenges we ran into

• Lack of Usability Testing
• Urgent needs for Frontend for better user flow on data streaming and suggestions provided by prompts inquired to Gemini Pro 1.5
• We needed to design an intuitive and informative visualization interface that accurately depicted the structure of conversations,
• Need to implement effective tuning algorithms to analyze transcribed text conversations and "small talk" conversation flow
• Ensuring the scalability and performance of GabbleGenie to handle large volumes of conversation data and user interactions posed challenges. Optimizing database queries, caching strategies, and server infrastructure to support concurrent users and real-time updates required careful planning and testing. Balancing system resources, response times, and data consistency was essential to deliver a reliable and responsive user experience.

Accomplishments that we're proud of

A rapid prototype for demo.

What we learned

The pivot of back end and front end is important because the end users need an understandable way to use the product. And the process of elaborating a potential social concern to a practical application. Translating theoretical concepts and potential social concerns into practical applications that address real-world needs requires the liaison between techniques and creativities. This whole hackathon deepened our understanding of user-centered design principles and the iterative nature of product development.

What's next for GabbleGenie

-Enhanced Conversational Analysis -Personalized Recommendations through supervised learning model -Incorporating real-time feedback and coaching features will enable GabbleGenie to analyze ongoing interactions and identifying opportunities for improvement or redirection. -Continuous Improvement and Iteration

Built With

• figma
• gemini
• python