My Buddy

Your all time talking partner

Comment

Inspiration

Many times, us teenagers and young adults have problems that make us feel sad or lonely. We need someone to talk to or get advice from, but often we can't find anyone. This makes us feel even worse. Sometimes we lose friends, and then we feel like we have nobody to talk to about it.

The app was inspired by this. It has three modes: friend, parent, and counselor. These modes talk to you, give you advice, and make you feel better and more important.

What it does ?

The web app can act like a friend, parent and counselor mode and can have a sweet conversation according to user mood, bothering, relationship status, etc. Each chat can be tailored for different moods and bothering, with support of all three mode: friend, parent and counselor in each chat conversation.

The chats are also stored in the database and can be revisited and the conversation can be continued whenever user feels to continue the previous conversation or want to visit the conversation he/she made previously

How we built it

To build our project, we went for MERN: Mongo DB to store chats and user settings, Node and Express for the backend, React JS for the frontend, and tailwind as a CSS framework. With some clever engineering, our chat system adapts to users' different moods, concerns, and relationship statuses.

When you start chatting, the first response you get is from the Gemini API. It uses a custom prompt based on what we know about you from sign-up and previous chats. We keep the conversation flowing smoothly using Gemini's features for multimodal and continuous conversation. When you come back to chat later, we fetch your entire chat history from the database and send it to Gemini API, along with a new prompt to keep things going.

To keep our server running smoothly, we only fetch the necessary data. First, we grab just the list of chats with names and IDs. Then, when you click on a chat, we fetch only the data for that chat, defaulting to the "friend" mode to keep things efficient.

Lastly, we make things easy with Google OAuth for logging in and signing up. It's a quick and secure way for users to get started, alongside the traditional sign-in.

To mention we also took input from our humanities professor to learn about human behaviour and emotions to tailor the prompt for specific emotions and users.

Challenges we ran into

Building this web app had its fair share of challenges. The file structure alone grew to over 25,000+ lines of code spread across more than 100+ files, making the project way more complex. Sometimes it felt like giving up, but we persevered. We worked hard, tackling every challenge and bug that came our way.

Our first big challenge was getting the Gemini to remember conversation history. It took us a good 2-3 days to figure it out. Then there were issues with fetching, loading, and saving data, but we managed to solve them all. Now, after all that hard work, our project is finally ready.

Accomplishments that we're proud of

We successfully added a feature that allows for multi-continued conversation. We also created prompts for different situations and modes, which we're really proud of. These were big challenges for us and the heart of our project.

Implementing these features is a major achievement for us. It was our first time working on a project every day for more than a month, and the complexity kept growing. Additionally, it was the first time we separated the frontend and backend to improve the experience, which makes us proud to have completed this project.

What we learned

While working on our project, we faced many challenges that helped us learn new things. One big challenge was adding Gemini with its ongoing conversation feature. It was our first time using Google OAuth, which was another learning experience. We also learned how to split tasks and work as a team effectively.

My friend and I had late-night meetings, sometimes coding for 3-4 hours, even though we live in different cities. Besides technical skills, we learned the power of collaboration and how to tackle long and complex projects during this hackathon.

What's next for My Buddy

There are various features we will be working:

  1. Support for photo: Users can upload their own avatar
  2. Prompt: Making prompt more stronger and implement latest version of Gemini model.
  3. A 3D voice talking feature: Implementing a voice talking feature in real time with bot expression shown by a three model.
  4. Fine Tuning: Using the feedback received from beta users, and after collecting data of different human emotions from professional counselor, we will fine tune the Gemini by using our own data set using Vertex AI.
  5. A Negative Score variable: If the data goes above this value, the user will be prompted to see an expert advice.
  6. Lastly to prevent spamming of account, OTP authentication will be added, with password changing options and modify the user profile.
Share this project:

Updates