Inspiration

We were inspired by the nostalgia of Tamagotchis and the idea of having a little companion by your side throughout the day. As students, we know how overwhelming it can be to keep track of assignments, quizzes, and deadlines. Canvas, in particular, does not always do a great job of making upcoming deadlines clear or reminding students when work is due. Important assignments can easily get buried among notifications and course pages.

We thought it would be helpful to have something sitting right on your desk that actively keeps you on track, reminds you of upcoming work, and encourages you when you're struggling. Rather than creating another to-do list that students have to remember to check, we wanted to build a companion that brings that information directly to them while making productivity feel more engaging and personal.

What It Does

Our project combines a physical robot with a web-based interface. By integrating with the Canvas API, it retrieves assignments, quizzes, and upcoming deadlines, organizing that information into an accessible dashboard. The physical companion displays upcoming deadlines, task progress, and an animated avatar with its own personality.

The web interface also features an AI chatbot powered by Google's Gemini API. Users can have conversations with the chatbot, ask questions, and interact with their AI companion. Currently, conversations use the laptop or phone's built-in microphone and speakers. We plan to integrate audio directly into the physical device so users can talk to their companion without needing another device.

We are also incorporating gesture recognition to make interactions more natural. Users can control music, navigate screens, and communicate with the companion through hand gestures. NFC tags provide additional functionality, allowing users to access a customization webpage or activate study mode to focus.

How We Built It

We combined embedded hardware, software development, APIs, and computer vision to bring our companion to life. The physical device uses an ESP32 alongside a screen, camera, and other electronic components. On the software side, we developed a web interface to display tasks and deadlines, integrated Canvas data, and connected Google's Gemini API to power the conversational chatbot.

We also explored computer vision and gesture recognition to expand how users interact with the companion. For the physical enclosure, we designed a custom CAD model and used 3D printing to create a housing for the electronics.

Challenges We Faced

One of our biggest challenges was integrating several different technologies into one cohesive system. Connecting Canvas data, synchronizing the web interface with the physical device, and implementing AI and gesture-based interactions required significant experimentation and troubleshooting.

Another major challenge was designing the physical enclosure. We had to create a CAD model that could properly house all the electronic components while considering dimensions, component placement, wiring, and accessibility. Turning that design into a functional 3D-printed enclosure was especially challenging given our limited time. We had to balance getting the design right with making sure we could actually print and assemble it before the deadline.

What We Learned

Throughout development, we gained experience with API integration, embedded systems, AI integration, computer vision, hardware-software communication, CAD modelling, and 3D printing. More importantly, we learned how to approach a project that spans multiple disciplines, break complex problems into smaller components, and adapt when individual technologies did not work as expected.

What's Next

Our next steps include integrating a microphone and speaker directly into the physical device, improving Canvas synchronization, expanding the Gemini chatbot's capabilities, and refining gesture recognition. We also want to improve the physical enclosure and make the companion more customizable along with decreasing the latency of data transferred.

Ultimately, our goal is to make staying organized feel less like a chore and more like having a little friend cheering you on.

Built With

Share this project:

Updates

Submission history