Inspiration

As a result of Quarantine, one issue that some people might encounter is a lack of interaction. Some might confront this issue by interacting with friends and family, but circumstances don't always guarantee that someone will be there. Some might confront it by interacting with pets, but some people might not have the space nor money to properly take care of their pet. And others might want someone or something to interact with that's a little more customizable, something a bit more... robotic.

What it does

Desky is a personal robotic friend that will keep you company while you are at your desk or table! Desky only requires a one-time investment, takes up very little space on your desk, and the only maintenance that you'll ever have to worry about is charging Desky up when it's charge is low. When Desky does not see a face, Desky will explore around your desk to provide a sense of liveliness to your work/rest area. If Desky sees someone who would like to interact with it(i.e. is facing Desky), Desky will track their face to maintain eye contact, and will react happily if pet. Desky also comes with a feature to add your own routines as well should you want to customize Desky further.

How I built it

This Desky prototype is built with an Elegoo Smart Car as the base, A Raspberry Pi 4 and a Movidius Neural Compute Stick (NCS) for computation, a Raspberry Pi Camera V2.1 for vision, and a 4" HDMI touch display to display Desky's face. I used a pre-trained face detection model (A big thanks to pyimagesearch for providing these files, as well as a tutorial on how to use them) using OpenVINO(a type of OpenCV that allows the use of the Movidius NCS) to detect a face and track it. This positional data was then calculated into movement commands send to the Arduino Uno on the smart car, which used FreeRTOS and task-based programming to interpret and execute these commands in real time.

Challenges I ran into

Learning how to use face detection model was difficult not only because it was a bit complicated to use, but also because the RPi camera auto-adjusts itself, so sometimes the model would not detect a face when it would a few seconds ago. The program would also take a while to load the model, so testing small changes became very tedious and took up a bunch of time.

Accomplishments that I'm proud of

Managing to be able to tie all 3 main software functions all together(Face detection and tracking, reacting to pets, and wander routine when no faces detected after a while), as all these components are very distinct, and I needed to be careful to connect them in such a way that they don't interfere with each other, and in a way that doesn't use up too many of the RPi 4's precious computational resources.

What I learned

I learned about using OpenVINO and consequentially OpenCV, the NCS, as well as FreeRTOS for Arduino and task-based programming.

What's next for Desky

Miniaturization, as well as potentially an API. This Desky prototype was made with very generalist hardware, which adds unnecessary bulk, complexity, and cost(which was fine when simply making a prototype). If another prototype were to be created, it would have more specialized hardware to cut down on everything that is unnecessary for a simple robot friend. In further development of the software, it should be possible to upload custom routines for Desky to make it more personal, as well as providing a fun mental challenge for those who would want to do so.

Share this project:

Updates