Take A Paws

Take A Paws
Problem Statement
Detecting Programmer Frustration
Detecting Programmer Productivity
Our Impact

Inspiration 💡

As developers, we often find ourselves stuck, frustrated, and confused when facing a large codebase requiring plenty of debugging or tough algorithmic problems. 83% of developers reported feeling burned out at work 🔥. Maybe what we need are some words of encouragement or reminders to step away to reset and recharge. Wouldn't it be even greater if that reminder came from a cute cuddly companion, looking out for our mental health 😼?

What it does 😸

Take A Paws is a VS Code extension, built to support developers' mental health. Your little kitten friend will respond to how you are feeling: Happy 😄, Sad 😔, Angry 😤, Confused 🤔, and even Surprised 😲. If you are coding with confidence, the code companion cat will cheer you on and help you maximize your productivity! Feeling a little frustrated? Your fluffy feline friend will detect this through your keystroke patterns and facial expressions and will suggest you to take a short walk 🚶‍♂️ to clear your mind so you can get back to coding with confidence 💻! Take A Paws takes in your facial expressions in a sophisticated weighting algorithm to determine the best way to support you.

How we built it 🛠️

Built using Typescript 📃, Node.js ⬢, and Python 🐍, Take A Paws utilizes the FER (facial expression recognition) library on a Flask endpoint and OpenAI with thoughtfully engineered prompts to provide you with meaningful encouragement with respect to your emotions while coding. The extension integrates with VS Code's API to monitor keystroke patterns, detect frequent pauses, constant undo and redo operations, and other frustration indicators. On the contrary, our extension detects when you are in a state of flow and provides positive reinforcement to keep you motivated and productive. Combining these metrics with facial expression data, the AI can accurately determine when you need a break or a boost of encouragement, and convey this message to the developer.

Challenges we ran into ⚠️

Combining facial recognition, character animation, and natural language processing within one VS Code extension was no easy feat. Our team was challenged with learning new languages, frameworks, libraries, and each others’ code in order to integrate all these pieces together 🧩. Ensuring real-time performance while processing facial expressions, and rendering the correct emotions and keystroke data needed a TON of testing and integration.

Accomplishments that we're proud of 🏆

We are proud to have created a useful extension for all developers that could use a reminder to rest their eyes 👀 or keep up with their good work! As we approach our final year(s) of undergrad, we are all excited to innovate and learn with our mental health in check!

What we learned 🧠

Akanksha developed her first VS Code extension and experienced prompt engineering for the first time 📃. Anna learned how to use Typescript integrated with Flask and Python and created her first mental health supporting tool 🌱. Nabil is proud of creating a backend with Node.js, integrating it with the OpenAI API, and crafting fun animations for the first time 🎉.

What's next for Take A Paws 😼

We wish to officially publish our Take A Paws VS Code extension so students and developers around the world can code alongside a perky personable pet 😸!