Gemma

Inspiration

As first year students in a new large campus compared to our high schools, we realized how lonely it can feel when in this new environment. However, we also realized that this feeling can still stick through later years. That's why we decided to build a digital companion that is accessible at any time.

What it does

Gemma's functionality is to act as an interactive, animated desktop companion. She directly lives and walks across your screen live to add life to your interface. Gemma's main function is to be a chatbot with customizable personality pre-sets: Best friend, co-worker and girlfriend. So each person can individually cater Gemma's personality to their own situation. When you speak or text to Gemma, she will respond to you with the personality you have chosen. Furthermore, you can also tell Gemma to play a song through text on Youtube music: "play counting stars" It will then open and play the song. Lastly,

How we built it

We built Gemma using Python and used PiP (picture in Picture) to install the libraries we needed, such as PyQt6, sounddevice, and pyttsx3. PyQt6 allowed us to create a transparent floating window that stays on top of other apps, similar to a small widget on your screen.

Challenges we ran into

One of the biggest challenges was getting audio recording to work smoothly without installation errors, which we solved by using Sounddevice. We also had to figure out how to make Gemma's transparent sprite window draggable and keep her chat bubble properly attached without disrupting her face.

Accomplishments that we're proud of

We are really proud of bringing a static pixel-art sprite to life as a walking desktop companion. Furthermore, we are also proud of integrating AI text generation, microphone recording, custom sprites into one smooth-running application.

What we learned

Building Gemma taught us how to use background threads (QThreads) in PyQt6 so the app doesn't freeze while waiting for the AI or recording audio. We also learned a lot about creating custom desktop overlays and handling live microphone data in Python.

What's next for Gemma

In the future, we plan on adding more animations, adding the male version of Gemma for diversity, make Gemma visually customizable, and give Gemma a voice by using text to speech.