StickSplit

StickSplit helps drummers see how they actually play by breaking each rep into finger, wrist, and arm activity over time—making technique feedback clearer, faster, and easier to improve.

Comment 1

About the Project

StickSplit is a computer vision coaching tool built for marching percussion. It helps drummers and instructors see how a player approaches the drum throughout a rep.

In competitive drumline, small technique differences can affect timing, sound quality, and visual uniformity. StickSplit turns subtle movement patterns into a clear visual breakdown so players can understand, compare, and improve their approach.

Inspiration

StickSplit was inspired by the reality of competitive drumline: consistency is everything.

Players are often told to use more wrist, less arm, stay relaxed, match the line, or keep their approach consistent. But those corrections can be hard to understand because they depend on feel, instructor feedback, and what someone can catch by eye in real time.

StickSplit makes those invisible technique habits visible. Instead of only hearing, “you’re too arm-heavy,” a player can see how their movement changes across a rep.

The goal is simple:

Make drumming technique easier to see, explain, and improve.

What It Does

StickSplit records a controlled marching snare rep through the app and analyzes visible movement over time. The app guides the user through a consistent recording setup so the drum, camera angle, and player position are easier to compare between reps.

After recording, StickSplit analyzes motion from three main movement sources:

  • Wrist
  • Arm
  • Bicep

These are motion-based estimates, not medical measurements or exact muscle activation scores.

The results dashboard shows:

  • Finger, wrist, and arm activity scores
  • A timeline graph of movement activity
  • Approach labels like Arm-Heavy, Wrist Break, Fulcrum Lift, and Lead by the Bead
  • Plain-language coaching feedback
  • A video scrubber for reviewing specific moments

A key design choice is that the finger, wrist, and arm scores are independent. They do not have to add up to 100. A player can have high wrist activity and high arm activity at the same time, which better reflects real movement.

StickSplit helps answer one coaching question:

Where does the stroke appear to be coming from?

How We Built It

StickSplit was built as a React Native and Expo mobile app powered by Firebase and a Python video-analysis service hosted on Google Cloud run.

The mobile app handles:

  • Authentication
  • Guided recording
  • Video upload
  • Session tracking
  • Results dashboard

Firebase handles:

  • Authentication
  • Video storage
  • Session documents
  • Analysis job documents
  • Result documents
  • Real-time result updates

Cloud run handles:

  • Video downloading
  • Frame sampling
  • Pose landmark extraction
  • Motion metric calculation
  • Approach classification
  • Writing results back to Firestore

Analysis Pipeline

1. Phone records video
2. video uploads to Firebase Storage
3. app creates a session and analysis job in Firestore
4. app sends job details to the Cloud run server
5. Cloud run downloads the video
6. video is sampled into frames
7. pose landmarks are extracted use python and mediapipe
8. motion metrics are computed
9. results are written back to Firestore
10. app receives live updates and displays the results

Computer Vision Approach

For Prototype 1, StickSplit uses MediaPipe Pose for hand and arm tracking.

MediaPipe Pose returns body landmarks for each frame. StickSplit uses relevant upper-body and hand points, including:

  • Shoulders
  • Elbows
  • Wrists
  • Index fingers
  • Pinkies
  • Thumbs
  • Nose

Each point is represented as a normalized (x, y) coordinate relative to the video frame.

To improve reliability, each landmark is filtered by confidence. If a landmark has low visibility, it is stored as missing data instead of being forced into the math. This helps prevent bad frames from corrupting the final results.

StickSplit computes:

  • Wrist break from the angle between the forearm and hand
  • Hand spread from distances between the thumb, index finger, and pinky
  • Finger activity from frame-to-frame changes in hand spread
  • Arm activity from elbow, shoulder, bicep, and forearm motion
  • Stroke timing from wrist movement peaks
  • Symmetry and consistency by comparing movement patterns over time

Scoring Model

A simplified scoring formula looks like this:

score = 100 × (raw - min) / (max - min)

Each movement source is scaled independently:

fingerScore = scale(fingerRaw)
wristScore  = scale(wristRaw)
armScore    = scale(armRaw)

The scores are not percentages of one shared total. Each score represents how active that movement source appears to be on its own scale.

For example:

Finger Activity: 72
Wrist Activity: 88
Arm Activity: 64

This means the player is using all three movement sources heavily, rather than forcing the numbers to add up to 100.

Challenges We Ran Into

One major challenge was defining what StickSplit was actually measuring.

At first, we described the app as measuring “muscle usage,” but video alone cannot directly measure true muscle activation. A camera can observe joint positions, angles, speed, and visible motion patterns, but it cannot prove which muscles are firing or how much force they are producing.

That pushed us to improve both our product language and scoring model. StickSplit now focuses on visible movement activity, movement source, and approach trends.

Another challenge was scope. Stick tracking and bead pathway analysis are exciting, but they are difficult to do reliably in an early prototype because drumsticks are thin, fast, and easy to blur on camera. For Prototype 1, we focused on the player’s visible body and hand movement first.

We also had to work around MediaPipe Pose limitations. MediaPipe has a dedicated hand model with more detailed finger landmarks, but we used Pose because it gives us the full upper body and basic hand points in one pipeline. That was enough for Prototype 1, but it means finger activity measurement is intentionally rough.

An additional challenge was figuring out the right camera angle. The analysis depends heavily on what the camera can clearly see, so the player, drum, hands, wrists, and arms all need to stay visible throughout the rep. Small changes in angle, distance, lighting, or drum height can affect the landmarks the system detects. This made us realize that StickSplit needs a guided recording setup so users can capture more consistent videos and get more reliable results.

Finally, connecting the full app-to-cloud-to-analysis loop required careful job states, result formatting, and error handling.

Accomplishments We’re Proud Of

We are proud that StickSplit turns a specific marching percussion problem into a product that feels understandable, useful, and buildable.

We are especially proud of:

  • Building a full mobile recording and analysis flow
  • Using Firebase for authentication, storage, jobs, and results
  • Creating a Python pipeline that processes recorded reps
  • Extracting useful movement signals from pose landmarks
  • Making results update live through Firestore
  • Designing a dashboard that feels like a sports coaching tool
  • Creating approach labels that match drumline instructor language
  • Using honest language around what computer vision can and cannot measure
  • Building an end-to-end flow from recording to analysis to results

The motion split idea is one of the strongest parts of the project. It gives players a simple way to understand a complex movement pattern: how much their approach appears to involve fingers, wrist, and arm.

What We Learned

We learned that the hardest part of building an AI product is not always the model. Sometimes the hardest part is defining the problem clearly enough that the technology can actually help.

We also learned that computer vision is powerful, but limited. It can estimate visible motion, but it cannot directly measure muscle activation from video alone. That distinction made the product more honest and defensible.

On the engineering side, we learned how to connect a mobile app, Firebase, and a Python processing service into one working pipeline.

We also learned how important calibration is when working with computer vision models. Small movements in the fingers, wrists, and arms can affect each other, so we needed to think carefully about thresholds that separate one type of motion from another. By tuning those thresholds, StickSplit can better estimate how much each movement source contributes to the overall playing approach instead of treating every visible movement the same way.

Most importantly, we learned that good product design means translating technical data into language users already understand. A drummer does not want raw coordinates or abstract joint angles. They want to know whether their approach looks arm-heavy, wrist-driven, inconsistent, or balanced, and what they can do about it.

What’s Next for StickSplit

The next step is to make the analysis more accurate, more visual, and more useful for real practice.

Future improvements include:

  • add more detailed finger movement metrics to better estimate finger control and activity
  • expand StickSplit to support more marching percussion instruments, including tenors, bass drum, and cymbals
  • adding stick tracking and stick pathway analysis
  • Improving per-hand breakdowns for traditional grip
  • Adding better video overlays and visual explanations
  • Adding side-by-side rep comparison
  • Allowing instructors to annotate reps
  • Tuning thresholds with more real player examples
  • Improving job progress updates during analysis

Prototype 1 focuses on visible movement activity from the hands, wrists, and arms. In the future, StickSplit could grow into a full technique review platform for individual players, instructors, and entire drumlines.

Our long-term vision is simple:

StickSplit makes technique visible, so drummers can build cleaner, more consistent reps.

Built With

Share this project:

Updates