Cranky Clippy

• 

  Cranky Clippy

Inspiration

We were inspired by the office pet Clippy. We noticed ourselves as well as many others struggling to stay focused and get work done, especially with all the distractions that come with the new digital age. So, we created this desktop pet to keep you from being distracted.

What it does

It exists as a little animated desktop pet in the corner of your screen as you do your work. It becomes happy when you stay on task, and angry when you get distracted. If you remain distracted for too long, it will get cranky and force you to get back to work by closing your games, distracting videos, and many other things on your computer that break your focus.

How we built it

We built this in python with tad bit of javascript, using Jev AI to detect programs and determine whether it is related to the current task you've decided to do. Using Gemini API, we were able to bring Cranky Clippy to life, allowing it to interact with the user using various dialogue that fit each situation. We also integrated ElevenLabs to give Cranky Clippy its voice. Cranky Clippy was designed in photopea, each of its facial expressions and animations hand-drawn using a trackpad.

Challenges we ran into

• As the front end for Cranky Clippy was originally coded in windows, some of the code only ran correctly on windows, failing to work on Linux that the back end was coded on. Specifically, the Win32 hotkey layer was causing the program to fail. To resolve this issue, we had to find workarounds and replacements that work on Linux.
• Jev AI was being way too strict, considering some applications that may be necessary for certain tasks as distractions. In order to resolve this issue, we added bunch of meta-data for different programs (hard coded directions that Jev AI follows) to check if what the user is doing is truly a distraction as well as asking Jev AI to consider whether what the user was doing was potentially necessary for the task. For example, it would consider OneDrive or a new tab in Firefox as unneccessary and a distraction for a science lab report. So, we made it consider broadly what each applications and windows implied rather than simply what they were.
• Jev AI was unable to correctly detect some of the applications because of sandboxing in Flatpak and Electron. For instance, Steam was being detected as chromium, resulting in Jev AI assuming it to be not a distraction. To resolve this issue, we made it detect processes inside of the application instead of what the application was.

Accomplishments that we're proud of

• The hand drawn Cranky Clippy design and animations that we made in photopea
• The decision making system that determines whether what the user is doing is appropriate for their set task or a distraction created using Jev AI
• The chrome/firefox extension that allows Cranky Clippy to switch tabs
• The application minimizing system
• The dialogue generator system made using Gemini API
• The text to speech that gived Cranky Clippy its voice made using Elevenlabs
• The demo video including a skit

What we learned

• How to work with AIs like Jev AI, Gemini, and ElevenLabs
• Different photoshop skills like using the pen tool to make a selection so that colouring in certain shapes were more accurate and efficient.
• Drawing precise lines and circles with a trackpad
• How to make a chrome/firefox extension
• How to use python libraries like PySide6

What's next for Cranky Clippy

• To make it run on other operating systems (MacOs and Windows)
• Better memory
• More interactivity (more ways to communicate with the user)
• More emotions (fun ones that are not necessarily required but increases the appeal of the software)

Built With

• davinci-resolve
• github-copilot
• github-desktop
• javascript
• opencode
• photopea
• python
• visual-studio-code