Caddie Coach

An Alexa+ caddie that learns how you actually miss, then picks your club, catches your patterns mid-round, coaches your swing from your camera, and turns your weak spots into a practice plan.

Comment

Inspiration

A useful caddie does more than report numbers. They remember how you have been playing, notice the mistake you keep repeating, and give you one clear suggestion before your next shot.

That was the inspiration behind Caddie Coach: a golf assistant that turns shot history and swing movement into practical feedback, rather than another dashboard to interpret. The goal is to connect what happens during a round with what a golfer can work on afterward, without requiring raw swing videos to be uploaded.

What it does

Caddie Coach combines round-aware coaching, browser-based swing analysis, and an end-of-round spoken recap.

As shots are logged, the caddie checks for recurring patterns and brings them up without waiting for the golfer to ask. Instead of reading out probabilities, it delivers a short, actionable observation:

"That's three approaches in a row short. Want one more club?"

For swing analysis, a capture page runs MediaPipe Pose directly in the phone's browser. Only six body keypoints per frame are uploaded; no video leaves the phone. The server uses those points to estimate swing tempo and identify movement patterns associated with coming over the top and early extension.

At the end of a round, Amazon Bedrock generates the text for a spoken recap. A template-based fallback keeps the recap available when generation is unavailable.

Together, these features create a feedback loop: log shots, notice patterns, review swing movement, and finish with a clearer idea of what to work on next.

How we built it

The project separates local motion capture, server-side analysis, persistent round state, and recap generation.

  • Browser-based capture: MediaPipe Pose processes the swing on the phone. The capture page extracts six body keypoints per frame and sends those coordinates to the backend instead of uploading video.
  • Swing analysis: The server derives tempo and movement indicators from the keypoint sequence.
  • Stateful coaching: Each logged shot triggers a check for streaks, allowing feedback to reflect the developing round rather than treating every shot as an isolated event.
  • AWS backend: AWS backend: Defined in one SAM template to run as a container on AWS Lambda with state in DynamoDB. A new-account plan limit blocked the deploy, so the demo runs locally agains
  • Generated recaps: Amazon Bedrock writes the end-of-round recap, with a plain-template fallback if generation is unavailable.

The key architectural decision was to keep raw video on the phone while giving the backend the compact movement data needed for analysis.

Challenges we ran into

The first challenge was making the assistant sound useful rather than merely analytical. An early version read out probabilities, but that did not translate into a clear next action. The more effective approach was to identify a relevant pattern and express it in one sentence.

Checking for streaks after every logged shot helped make that shift. The assistant could surface something the golfer might have overlooked, rather than waiting for the golfer to formulate the right question.

A second challenge was designing swing analysis around a small set of body keypoints instead of server-side video processing. This required focusing on specific movement indicators rather than presenting the system as a complete reconstruction of the swing.

Another design consideration was keeping the recap from depending entirely on a successful generative-model response. The template fallback provides a simpler path when Bedrock generation is unavailable.

Accomplishments that we're proud of

  • Turning shot history into proactive, concise coaching instead of a stream of statistics.
  • Keeping swing video on the phone while supporting server-side movement analysis.
  • Connecting round tracking, swing feedback, and a spoken recap into one experience.
  • Giving each AWS service a defined role: Lambda for backend execution, DynamoDB for state, and Bedrock for recap generation.
  • Including a non-generative fallback for the end-of-round recap.

The part we are most proud of is the change in interaction: the golfer does not need to ask, "What pattern am I missing?" The caddie checks for it as the round develops.

What we learned

Building Caddie Coach taught us that more information does not automatically make a better assistant. The important step is deciding which observation matters now and communicating it clearly.

We also learned to separate measurement from explanation. Shot history and body keypoints provide the underlying signals; generated language helps communicate a recap. Keeping those responsibilities distinct makes the experience easier to reason about.

Finally, privacy influenced the architecture from the beginning. Processing video locally and transmitting only selected keypoints shaped what the backend needed to receive and what the analysis could reasonably claim.

What's next for Caddie Coach

The next step is to test the coaching and swing indicators with more golfers and a wider range of phone placements, lighting conditions, and swing styles.

Planned improvements include:

  • Validating swing indicators against feedback from golf instructors.
  • Adding clearer capture guidance and confidence checks for unreliable pose tracking.
  • Personalizing coaching using a golfer's longer-term shot history.
  • Connecting recurring on-course patterns with targeted practice suggestions.
  • Refining the recap so it prioritizes the most useful takeaways rather than summarizing everything.

The goal is to make Caddie Coach better at the thing that inspired it: remembering the round and offering the right observation at the right moment.

Built With

Share this project:

Updates

Submission history