Tsundere AI Girlfriend

Inspiration

I was inspired by wanting to make a companion to talk to when doing work (in case there were no friends to talk to)

What it does

This is a unity file where you can open it and it shows MaryBot. By pressing the mic button, you can ask a question or start a conversation where she responds in turn as a tsundere does.

How we built it

I used a collection of open source resources in order to put this project together. First, Daz3D was used to generate a custom 3D model which is then exported into Unity and paired with Mixamo for animation. There is a button which records a specific length of sound then utilizes Whisper for speech recognition. That output string is set as input to a locally downloaded LLM (llama as it was small) and an output is generated as a string. This string is then parsed into phonemes and output as sound using Jets Text-to-speech model in Unity Sentis and at the same time, blend shapes from the Genesis 8 model from Daz is modified for simultaneous mouth movement

Challenges we ran into

The most difficult parts was that there were so many steps such as integration with Unity that made this difficult, especially for a specific project like this

Accomplishments that we're proud of

The most difficult parts was that there were so many steps such as integration with Unity that made this difficult, especially for a specific project like this so getting it done was an accomplishment

What's next for Tsundere AI Girlfriend

There are always more things to expand on like: better speech recognition, higher quality LLM access, better blending of features and emotions, better computer

Notes

Proprietary Assets by Daz3D means that the model could not be uploaded and so whilst the Github code itself is not a functioning standalone repository, it serves as a guide to the process and setup of what to do. Or contact for more info/steps/resources