Due to the high risk of transmission of COVID-19, trick-or-treating during Halloween was not feasible this year. I have a little brother who was very sad about not being able to trick-or-treat, so I wanted to do something that would cheer him up, and at the same time address the problem at hand. That's how I came up with the Ferrero Freak. It's a Halloween themed, Arduino based, non-contact, autonomous Ferrero Rocher dispenser that uses machine learning models in order to evaluate whether the trick-or-treater is wearing a mask. Accordingly, it will dispense chocolate to reward the little trick-or-treater or pull a small (but safe) prank on them to encourage them to wear a mask. If properly used, the Ferrero Freak is capable of making socially distanced and/or automated trick-or-treating entertaining, viable, and most importantly, spooky!

What it does

The Ferrero Freak is a spectral Ferrero Rocher dispenser that combines the power of Arduino hardware, Tensorflow.js machine learning, and Python scripts to reward masked trick-or-treaters with high quality chocolate while allowing the operator to dispatch devious pranks on unsuspecting unmasked (or even masked) trick-or-treaters. It can operate autonomously or be controlled remotely via Python.

How I built it

This project is actually pretty complicated for how simple it looks. First I created the code in C++ to control the Arduino microcontroller and hooked up a few servos (one to control Ferrero dispensing, the other to control the prank mechanism), an ultrasonic distance sensor (to trigger chocolate dispensing), and an active buzzer (for some custom spooky sound effects). I engineered a sliding mechanism to dispense the chocolates and a pulley mechanism for the prank.

Next, I trained an image classifier machine learning model through Tensorflow in order to detect whether the trick-or-treater was wearing a mask. Through Tensorflow.js, ml5.js, and p5.js, I created a browser based interface for monitoring the webcam attached to The Ferrero Freak.

Afterwards, I used pySerial to make a simple user interface for the operator to optionally give custom commands to the Arduino (such as pranking them or playing a spooky tune). Finally, I anchored the Ferrero Freak to a PVC pipe and made it look sp00ky with a ghost costume. This made it look pretty cool and masked all the hardware and unaesthetic wires inside.

Challenges I ran into

I am completely new to the Arduino platform, so learning how to use all the hardware was pretty difficult for me. Engineering the trap pulley mechanism and candy dispensing mechanism with access to very few materials was definitely quite challenging, but I made a fully functioning device with everyday objects. Over the course of this project, I really grew to appreciate the Arduino platform and am really hoping to take my next projects to the next level by incorporating more advanced hardware and getting some more practice with Arduino in the future. Training the machine learning models in a short period of time was also tricky, but in the end it managed to work quite well.

Accomplishments that I'm proud of

I'm pretty proud of independently creating The Ferrero Freak from scratch in about an 18 hour time period. Building everything, debugging, and even learning Arduino was a pretty hefty task. But in the end, I was able to make something that I am proud of. I hope that when my brother wakes up in the morning, he will think that what I built was pretty awesome!

What I learned

I learned a lot about Arduino, using hardware, and C++. I've never really programmed in C++, nor had any significant exposure to hardware prior to this project. However, I'm pretty experienced with JavaScript and machine learning, so that part of this project went quite smoothly.

What's next for The Ferrero Freak

I'm hoping to save all of the code for The Ferrero Freak, add some more advanced hardware and sensors, and take this project to the next level in Halloween 2021. Perhaps the non-contact nature of this device can also be used in other solutions during the COVID-19 era.

Built With

Share this project: