Inspiration
We have all been fascinated by the recent breakthrough in conversational NLP, the ChatGPT model by OpenAI. Spanning multiple domains, the distinguished performance of the model inspired us to build something that could fit into our daily lives. Mitram comes from Sanskrit, meaning a true friend. Through Mitram, we've tried to build a speech-based conversational agent that can help us solve day-to-day queries and give the continuous feel of a friendly presence.
What it does
- Mitram is an AI that can have a conversation with you and even answer your questions
- Mitram can take text-based as well as speech-based input from the user
- Mitram then responds verbally to anything the user says / type
- The conversation is transcribed in the chat box so the user can see the history.
How we built it
- First, we set up a baseline web application to interact with the AI
- We then ask the user for input. If the input is given verbally, we live transcribe it using AssemblyAI to convert it into text
- Then, we relay the text to ChatGPT to generate a contextual response
- Finally, we utilize the speech-tts API to create an audio file to provide verbal output to the user.
Challenges we ran into
- Choosing the optimal API for each subtask
- Figuring out how to use all the APIs
- Hosting and deploying the web application.
Accomplishments that we're proud of
- Connecting all the APIs together
- Integrating frontend and backend of the web application.
What we learned
- Project implementation/development skill
- ChatGPT, React, node.js, AssemblyAI.
What's next for Mitram.ai
- Using a more modern API for text-to-speech conversions, such as Uberduck
- Fine-tuning a conversational agent to perform context-based discussions.
Log in or sign up for Devpost to join the conversation.