Leo: co-pet virtual assistant

Interaction with Leo
Chat interface
Caption screen

Inspiration

We are inspired by classic workspace companions like Microsoft Word's Clippy and its modern counterparts like VScode's pet extension. We believe that an animated companion can not only spice your workspace up a bit but also be helpful.

What it does

Leo is a Gemini-powered collaborative pet assistant on your desktop that you can interact with. You can chat with Leo, ask Leo about something on the internet, or ask Leo about something on your current screen. It will roam your desktop and provide you with the interactivity that those tiring (or boring?) screens need. In a nutshell, it is a fun companion (with the cost of a paw of your productivity).

How we built it

We built Leo using ElectronJS as its front-end and Python Flask server as its back-end. Leo is powered by Google Gemini. We build specific prompts to power Leo's interactivity. Then mostly Python to get Leo access to some of your workspace estate like folder structures, screen, etc.

Challenges we ran into

As we are building a desktop app, compatibility between operating systems is a major challenge. We need to run multiple checks on different devices to make sure that Leo works. Some functionality only works or only works best in an OS but not for the other, but we are working to help Leo shine wherever it is.

Accomplishments that we're proud of

We are proud that we've successfully created an assistant that is packaged in an unconventional way. We are providing you with a unique experience to interact with an assistant, straight from your desktop. Of course, with hopes that this would spark fun activities while you work.

What we learned

We learned a lot about desktop APIs, especially with Electron. It is really fascinating and refreshing to work with desktops as people tend to go for websites nowadays. We learned a lot about compatibility between desktops of different OSes and also how to maximize the potential of a desktop app.

What's next for Leo: co-pet virtual assistant

We are planning more features and more interactions for Leo. Now, Leo is not very versatile. But as it is powered by Gemini, its powerful potential awaits. What we are planning for Leo: