As part of the dance community, it is often difficult to record and routines and pieces due to a stationary camera. When practicing, dancers often leave the camera frame and the center of the camera is not constantly focused on the dancer. You do not always have a cameraman to record your dance piece.
What it does
The camera uses body and facial recognition so that you can keep the camera following your body movement and keep you centered in the frame. So that during your routine, the camera will pan and follow your movement to keep your body focused and centered in the video frame.
How we built it
We split the project into 2 parts: hardware and software.
Our software team was in charge of creating a wireless media buffer for the GoPro5 and stream the footage directly to our computer. Then, our next task was to take in each frame and apply Machine Learning facial and body recognition algorithms to detect the person of focus in the frame. It draws a rectangular box around each body detected in the frame, though we set constraints to prevent detection of people in the far background. Thus, we are able to stream a live feed from a GoPro 5 to our laptop and the footage presented has the body recognition rectangles drawn around the people in the frame. Then, we took the boxes drawn to determine a person's location and wanted to connect our panning arduino motor to follow and track a person's movement, while still streaming the live footage with the ML recognition running concurrently. The arduino calculates the movement of the body recognized and adjusts the motor sensor accordingly in real time to track and pan the body movement.
Our hardware team was in charge of working with the Arduino kits. the grove add-ons to the Arduino, and the GoPro. The first Arduino board has a servo motor, which contains a shaft that can be positioned at any specified angle. The first objective was to move this component according to a serialized input angle and direction. We then created a python file containing all the methods that the GoPro script can call to connect to the Arduino board and to rotate the servo. The hardware team decided to take it one step further and add one more Arduino board containing an LCD Display that would allow the dancer to keep track of the duration of the dance. Once we had this working, we decided to add one more final feature to it. We added Google's Cloud Speech API to enable voice recognition to start and stop the timer sessions. As a way of communicating to the dancer that the recording was about to begin, we also added in the Speaker component of the Grove Arduino kit, which makes three beeps when the timer is about to start.
Challenges we ran into
None of our team members had previous hardware experience. So, we had to learn how to everything works from scratch. Also, we struggled with making sure all the connections and ports were properly set up to the hardware. We were able to overcome these obstacles through a lot of research and asking around. Also, we tried to incorporate Google Home minis but the devices needed special permissions from the Georgia Tech Technology department, which would not look at the request until after the weekend.
Accomplishments that we're proud of
We are proud of being able to incorporate hardware into our project, given that we had no previous experience in the field.
What's next for Follow Focus
We hope to expand the number of ways that Follow Focus could be applied. Our vision is to help people of all ages and sizes to get moving through aiding them in all sorts of exercises, whether it be dancing, push-ups, squats, or even running.