The FreeCam System

Tired of always sitting in one place during meetings? Do you wish wish that you could stretch your legs a bit and walk instead of stiffly staring at the camera in front you? Introducing FreeCam.

Comment

Inspiration

I am someone who can't sit still for long. When at college, I enjoyed the rush to different classrooms as compared to being let out of the classroom only during lunch and washroom breaks while at school. However, Corona wrecked everything. Now with online classes, my movement is limited to my chair. On top of that, whenever my face sways out of the camera during exams, the teacher immediately calls out, which I find really annoying. There was also this one time where I accidently left my camera on and something rather embarrassing happened making the entire class laugh. I wanted to make something that can handle these problems. Even the teachers has to stand still or sit in a chair while taking class online.

What it does

My system is a camera that automatically tracks one's face as the person moves around. Even the side of the face is tracked. At the same time, if no face is detected for some time, then the camera turns itself off.

How I built it

The camera is controlled by two servos connected to an Arduino which is connected to the laptop. The camera is also connected to the USB through a separate USB port. Using OpenCV and deep learning, I extract the position of the face in the frame. These position coordinates are then serially sent to the Arduino. The Arduino calculates the angle the servos have to turn for the camera to center the desired coordinates and controls the servos. The OpenCV app outputs a window which I capture using OBS studio and enable as a virtual web cam.

Challenges I ran into

There were lots of challenges in making this system. The biggest one was the lag between each frame. Since the calculations were done for every frame, there was a lag. Changing it to every fourth frame didn't help as now it would lag every fourth frame. Moreover, the timer program for turning off the camera was to run parallelly to the main program. The Arduino would keep on turning the servos too muck or too fast making the face detection algorithm go crazy. Finally all of this was implemented through multi threading. The angle calculation was offloaded to the Arduino and a PID algorithm was implemented to fix the speed and overshoot issue.

Accomplishments that I am proud of

I am just proud of the fact that I actually managed to finish this project on time. As someone who usually deals with hardware, having to do this much programming was a new experience for me.

What I learned

I learned that something called multi threading exists, how it works, how it is very different from multi processing and implemented it in my code to smoothen operation. I also learned how to implement PID in C without using any extra libraries. I also in the span of this hack derived the calculations for converting the (x, y) coordinates visible in the frame into angle values for the servos.

What's next for The FreeCam System

For now, the system relies on keyboard input to operate and OBS studio to be seen as a web cam. I plan on resolving both these issues by writing additional scripts to make the system wireless and display itself as a virtual web cam. The PID parameters have also not been fine tuned causing a lot of shaking which I plan to fix later.

Built With

Share this project:

Updates