Vivy.ai

Vivy is a smart, creative, clever AI that wants to talk with you!

Comment 1

Inspiration

We've always loved AI, and GPT-3 was an AI algorithm we've wanted to utilize for some time now. Then we learned about the power of Assembly.ai and it clicked, let's make an AI bot to talk with on discord. We also wanted to base it on a character from an Anime we both enjoyed, Vivy: Fluorites eyes. Vivy is a robotic AI who loves to sing, and we wanted to bring her personality to life in our bot. Finally, a huge inspiration for this project was the GPT-3 interview from Open-Ai, we wanted to try our hands at creating something similar. https://www.youtube.com/watch?v=PqbB07n_uQ4

What it does

  • Vivy is an AI that will have a conversation with you on discord
  • Vivy will join a voice channel and then listen to the user
  • Vivy then verbally responds to anything the user says.
  • The conversation is transcribed in a text-channel so you will never forget it!

How we built it

  • First, we set up a basic discord bot to retrieve audio into .wav format
  • We then send the wavfile data to Assemblyai and then wait for it to be processed, retrieving the speech-to-text
  • Then we sent the text to gpt3, including our prompts, to generate a contextual response
  • Finally, we used utilized Google's text to speech along with GPT3's response to create an MP3 file to playback to the user on discord

Challenges we ran into'

  • Transcription time
  • Figuring out how to use all the API's
  • Incorporating Vivi's traits for GPT3 to learn from

Accomplishments that we're proud of

  • Saving audio from discord call
  • Generating an AI response to questions
  • Creating a bot that can communicate via voice call
  • Transcribing discord call

What we learned

  • Project implementation/development skill
  • AssemblyAI, GPT3, Discord.js, Google Cloud

What's next for Vivy.ai

After creating Vivy.ai we realize the potential it has, and a couple of routes we could take with her;

  • Train the AI model with Vivy context to more closely resemble her
  • Create a 3d model of Vivy, like vtubers, and have her speak
  • Train her to sing

Bonus

Alt Text

Built With

  • assembly.ai
  • discord.js
  • google-cloud-text-to-speech
  • open.ai
Share this project:

Updates