Mash

Customer service agent which supports text and Voice!

Comment

Inspiration

It all started on 21st December — my birthday My sister ordered sneakers for me from a company form India , they used Whatsapp to update about about the order status. They told me that it would take 3 days (24th Dec) on Whatsapp. The order didn’t arrive. So I messaged their WhatsApp number to ask about the status and I got this exact response:

"I may struggle with responding to open-ended or freeform text, but you can use buttons to interact with me more easily." then I clicked “Talk to Customer Service.” It took exactly 1 hour and 30 minutesfor my query to get to a real human who then solved my query and said I should get them in 48-72 hours but guess what? I had to go through the entire process again from scratch. This is why I build a smart voice Assistant which not only supports text but voice as well.

What it does

Mash is a Smart Customer Service Agent that responses to people to peoples queries in Seconds and does not take hours to respond. It makes the customer Experience much better and helps your Brand look Cool and Caring.

How we built it

We used Python and FastAPI as backend and in the frontend we used Next.js with React and Typescript. It started with making a SPEC.md of the whole project. We started with the backend and made sure first the functionality was in place. We used the Gemini-3-flash-preview model as the brain. We first built the chat functionality first, We got the Meta WhatsApp Business API(free) then integrated it with the Gemini-3-flash-preview model and first got the text working. then we made the front end and deployed the backend on Railway and frontend on Vercel. We then integrated the Voice-notes using Deepgram for STT and then sending that message to Gemini model and receiving the response from the model and then sending the message by Whatsapp Webhooks . We also added PostgressSQL and Redis in this for Scalability.

Challenges we ran into

This was my first time building a full project. What I mean is this is the first project I made that I personally think is the best! of my current abilities. We ran into a lot of challenges . 1) I had no idea how to get a Whatsapp business API. 2) I had worked with Gemini API key before but had never fully understood it I mean I just vibe coded it but this time I had to search thru the Docs 3) I had never used PostgreSQL before and Redis before I still have to learn a lot about them but at least I got started. 4) I had to learn x*typescript, NEXT.JS* for the frontend because in my previous hackathon the frontend was not really good. 5) I had to understand how do pipelines work and how to make systems that scale. Still need to understand more about them. 6) Orginal Idea was to make a Calling Agent but, had to backtrack on it for now as I did not find a way to make for free. Had to change plans and understand what can be made out of the current situation. Overall this really taught me a lot of stuff.

Accomplishments that we're proud of

We Started with nothing but just the idea and the problem we hoped to solve When we asked the llm how should we go about solving the problem and the idea we are trying to build it gave us a SPEC.md of the solution of which we did not know probably 30 % of the stuff. We started build it thinking if it we are even able to get a response from the gemini model on our Whatsapp it will be so cool and we will think the project is complete. We were really happy when the bot chat got blue ticks. The moment that we were overjoyed by when we sent the voice note and we got a perfect response back!! from gemini **. Personally I thought we would **not have enough tokens in the free API key to support Voice but it somehow worked!

What we learned

We learned about how to structure big project, how to think what the website user would like and how would user like it was a very new perspective for us we still need to develop that further but it did start form this project. On the technical end I learned how to use PostgreSQL and Redis, learned Typescript and More about API Key integrations, how to work with gemini-3 better learned about team structure and management and how to think in terms of system pipelines. Deployment I think this is the first time I have ever deployed a complete project how to use Railway and vercel.

What's next for Mash

I think the next step would be Integrating Calling Agents. I also believe that we still have to improve our User Experience. I believe Mash could really be a full fledged Customer Service Agent that can impact Startups in making there customer Experience far better.

Built With

Share this project:

Updates