Pymon

Pymon! The smartest Teyvat companion, right amongst your friends!

Comment

Inspiration

When I join a fandom of any kind, be it a game or a show, or anything else, I usually find it hard to get into it and have answers to my questions when I don't have a friend that is also part of that fandom, and as an introvert, I don't feel comfortable asking fandom groups directly, scared of judgement.

So I end up looking around the internet looking for answers, wishing I had someone to talk to comfortably; isn't there a better solution than an AI?

What it does

Pymon is a discord bot that answers any question you may have about the game, it can also give advice on how to enjoy the game to the fullest and build your teams and characters! While pymon is genshin focused, it can easily be scalable to any fandom with a few minor changes in its roleplay prompt, and the chosen (public) database file.

The reason why we chose discord is pretty simple, it's the best place for an assistant for fandoms, let me explain: Many fandoms have reddit pages and fandom wikis, but when it comes to coming together and just chatting about the media they are a fan of, Discord is the universal choice. On top of being a chatroom based server where people gather, it is especially convenient for games, as people tend to keep discord open while gaming (chatting either on text channels of voice channels) everyone can ask Pymon anything, and everyone can benefit from seeing the assistant's responses and learn more.

For example, if you and your friends are on voice chat and are chatting about the game's lore, but all of you are not sure about something, like the origins of a character, just ask Pymon directly on the server where everyone is gathered and can make Pymon join in on the conversation!

Pymon is triggered by pinging her, or starting your message with the phrase "tell me pymon". Replying to a Pymon previous message will keep take that one message into her history, and trigger her to respond as well.

How we built it

Pymon is built on Python over many things:

  • Discord API: what makes the assistant an interactive discord bot in servers amongst communities.
  • Gemini API: the central brain of Pymon, Gemini 1.0 Pro is what makes Pymon think and speak!
  • Qdrant: the vector database containing all of genshin impact's wikia's page data, for Pymon's RAG capabilities. The VDB's data is made from the public fandom wikia's database.
  • DuckDuckGo search: keeps Pymon always up to date with the latest news, adding one more layer to Pymon's RAG.
  • HuggingFace Hub: the vector database needed embeddings generated form a sentence similarity model, that was achieved using open source models form the HuggingFace Hub.
  • Docker: what's a bot if it's not deployed? safely kept in a controlled container, the entire Pymon project has all the necessary code to be contained and ready to deploy with docker.
  • Love and passion: the love and passion for both AI and games (and all the other fandoms we are a part of) and bringing communities together with some fun mixed with convenience.

Challenges we ran into

  • Time constraints: we have full time 9-5 office jobs, leaving very little time for development, but we made it possible.
  • Money: no free deployment options, or the budget to deploy (in some cases even some websites didn't allow us to pay for it for unknown reasons) so Pymon is currently deployed in a 15 years old laptop under my bed!
  • Building a database: parsing a database for the VDB from an xml file wasn't an easy feat, it leave a lot to be desired but Pymon can understand the data well even though it is not structured well!
  • Ressource restrictions: we had big dreams for the VDB, for example using a multilingual model for the best multilingual capabilities, but the bigger the model the more ressource needed, and with what we have the longer it will take Pymon to reply, so we went with the smallest model we could find, that even though it wasn't meant to be multilingual, can still manage with the other languages fields in the VDB data.
  • Deployment: as mentioned previously, Pymon is deployed locally in an old computer, that is on 24/7. But we had Pymon disconnect many times due to the internet instability in my home.
  • Hallucination: like every LLM, Pymon suffers from hallucinations sometimes, making up facts or mixing up information provided, we adjusted this as best as we could with the time limit, but with more research on the link between RAG and her behavior, this problem can be fixed in future versions. So far testers have tried tricking her and convincing her of false info, to which she simply corrected and stayed true to facts.
  • API issues: some 500 error codes and RECITATION error from google's side were detected, as well as a 2000 length error message from discord (this one was fixed).

Accomplishments that we're proud of

  • Successfully built a functional Discord bot: Despite everything Pymon is real and is there kicking, and is being helpful and fun companion to travelers who have tested her.
  • Positive user feedback: Pymon had a few testers who played a huge role in improving her behavior and her answer quality, which always ended in user satisfaction. Pymon's personality also made the testers have a lot of fun, trying to trick her, or to get her to roleplay along.
  • Increased engagement within the fandom: Helped bring people of a discord server together and play around with Pymon! Also improving the players' engagement with the game (gameplay and lore alike).

What we learned

  • Challenges of working with large datasets: Building and maintaining a comprehensive database requires ongoing effort and optimization. Not only parsing it from an XML file of web pages, but also keeping it up to date.
  • Prompt engineering: One may think RAG and prompt design is easy, but for an intricate assistant, it gets difficult to make the assistant understand who it is, while also using the provided RAG data intelligently, without making up fake information.
  • The power of open-source: We take it to heart: Pymon is open source and always will be. Our goal was to bring communities together, and the best of all communities is the open-source one for sure. Pymon wouldn't be here without open-source models, open-source database, open-source frameworks, open-source programming language, ...etc. Nothing would have been possible without it, and we are happy to be part of this community, and contribute to it.

What's next for Pymon

More improvements! Better quality data! More fun functionalities! More interactions! So many great ideas for Pymon, the first ones being improving Pymon's answer quality, with a better database, and even better prompting. Adding the chat feature was a big one we wanted to implement, but with time constraints, we preferred improving Pymon's core features and answer quality as best as we could, before taking on a bigger bite. The chat feature we wanted to implement was based on discord's thread feature, and keeping RAG's context retrieval (vdb semantic search) data as well for ongoing chats as long as the thread was still alive in the server. Making more interactive features is also in our plans, such as making the Pymon users create their own data in Pymon's database, and use that data to showcase their game progress and achievements, and even make them interact (like mini contests).

Implement the VDB population in the source code, so far the VDB is being segmented and populated through jupyter notebooks (that are available in the repo), which we plan on implementing in the source code for easier access.

There are so many things Pymon could use to be an even better bot and companion, and we will work towards that goal!

Built With

Share this project:

Updates