Game of Squid

Official teaser image
Early CAD model of the turret
MoveNET pose detection on the webapp
Web App landing page

Inspiration

We really liked this popular show and thought to ourselves, “those games looked fun!”. We made our dreams come true by making the hottest toy of 2022, what we are calling the ‘Game of Squid’ (no copyright intended). It’s all the fun of the show’s hit game and cultural classic ‘Red Light, Green Light’, but with half the bloodshed!

What it does

Game of Squid is a hardware-software system that runs the classical game “Red light, green light” from a popular show. The rules of the game are simple, when the game master calls “Green light”, players are allowed to move from the start line towards the finish line. Whenever the game master calls “Red light”, players must immediately stop in their tracks, if they are caught moving, they are eliminated from the game.

Our product allows players to feel the thrill of playing in a high-stakes game of “Red light, green light” by recreating the AI-based movement detection using machine learning and by simulating the gun shots with a DIY nerf gun mounted on a turret. The game supports up to 7 players with one person selected as the game operator. The game operator controls the laptop and decides when to switch the light from green to red and vice versa, everything else is completely automated. When the light is red, the web app kicks in to determine any player movement. If detected, the algorithm will calculate the necessary angle of the turret and send a signal to fire. Players who are hit by a nerf dart are thus eliminated and the game continues - or at least was the original plan. Due to one of our motors burning out and numerous futile attempts to revive it, we switched to using a laser pointer to have a similar effect.

How we built it

Our product is split into two sections, software and hardware.

On the software side of the product, we developed a web app written in Typescript to run the game and also perform real-time pose detection. Using Tensorflow’s MoveNet library, we integrated it into the React app to detect the poses of each player in real-time. When the light turns red, after a small delay, the app takes a snapshot of all the poses within the frame. This is used as a reference point to determine if the player moves during the red light period. While the light is still red, the app actively checks the new poses of each player and if any component of a pose exceeds a predetermined threshold, the player will be eliminated.

If the player is to be eliminated, the app then calculates the angle required for the turret to hit the player. This angle is sent to the Node.js server using Socket.io, which is then piped over to the Arduino which controls the hardware.

On the hardware side, darts are loaded into the magazine chamber and fed into the firing mechanism by a rack and pinion mechanism connected to a servo. A pair of drone motors are used as flywheels in order to fire the dart over 30m. The firing mechanism is positioned on top of another high-torque servo that acts as the turret mechanism allowing the servo to aim at the target. As fore mentioned, this was replaced with a laser pointer indicator. All these components are interfaced with an Arduino Mega.

Challenges we ran into

Software component: One of the problems we faced was having to find the distance of the players from the turret in order to accurately determine the shooting angle. We first thought of using a LIDAR sensor to map out a 3d space of the players, however, this seemed to complicate a simple problem at hand. Afterwards, we discovered that we could map the distance between two points on the pose to the distance from the camera. We chose the distance between the shoulders as it was the most reliable, then we measured points from different distances from the camera and created a general formula for estimating the distance. While playtesting the game, we realized that if players were standing too close to each other and one person moved, the pose detection would detect both players to have moved making it unfair to the person who didn’t move. We believe this is a limitation of the machine learning algorithm as it can only infer so much information from a 2D stream of images. To combat this issue in the future, a LIDAR scanner would be helpful to isolate the targets.

Hardware component: Originally, our group wanted to use a medium stepper motor as the turret mechanism. However, due to the motor we ordered being a pancake motor instead, we decided to abandon the idea and use a high torque servo motor instead. Similarly, due to our original idea of using a stepper motor, we planned on using a lazy susan bearing as the pivot mechanism. The mechanism though was on the heavy side and the bearings were not sufficiently greased causing there to be an excessive amount of torque. Overall, the hardware went pretty much every where, when debugging a problem a motor burnt out and lowered morale and everything went wrong afterwards. Also our motors may have revived themselves but then a stepper motor broke. Love hardware hacks!

Accomplishments that we're proud of

Developing a hackathon project that combines both hardware and software was no easy feat. One thing that helped us succeed in completing the project was our ability to collaborate and persevere through hardships. From having a solid plan going into the hackathon, we were able to delegate the work to be done efficiently across each team member. Each of us focusing on our specialty helped immensely.

What we learned

Game of Squid is very fun but can be painful. I hate the Arduino serial port, nerf darts hurt more than they thing, and monster energy is not great for your health.

EDIT: worked 10 mins before the deadline