PlaySmith

• 

  Overall product view with canvas and simulator

• 

  Learn more about each player's route through AI insights

• 

  Visualize your play in a 3D environment

• 

  Download a clip of your play

💡 Inspiration

Coaches still sketch plays on whiteboards—and players squint and guess. We wanted a tool that understands those scribbles instantly, tests them against a real defense, and turns them into something you can see, measure, and share in seconds. PlaySmith closes the loop from idea → simulation → feedback → highlight.

💻 What it does

PlaySmith lets you draw a play on your phone (or scan a coach’s board), auto-parses it into structured Play JSON, and simulates vs. defense with tunable speeds and policies. It then produces a highlight clip and an AR walkthrough, and surfaces AI coaching metrics (spacing, advantage windows, drive-lane openness, tempo). You can save plays to a library and use Gemini embeddings to find similar pro sets for inspiration or scouting.

⚙️ How we built it

🧠 Challenges we ran into

• Coordinate & timing normalization: Converting freehand canvas paths into meter-based, monotonic keyframes for both O and D without drift.
  
• Reliable parsing: Making auto-parse resilient to label noise (“Offense” vs “O”), origin flips, and irregular stroke spacing.
  
• Media generation constraints: Handling model rate limits and strict MIME modes while keeping a graceful local fallback for thumbnails/clips.
  
• Real-time insights: Computing coach-useful metrics fast enough to feel interactive.

🏅 Accomplishments that we're proud of

• End-to-end loop: Draw → parse → simulate → metrics → clip/AR in one cohesive flow.
  
• Actionable analytics: Spacing, advantage windows, drive-lane openness, hull-space index, tempo—plus AI coaching in validated JSON.
  
• Similarity search: Gemini embeddings power “Find Similar Plays” across a growing library.
  
• Polished UX: Sport-aware backgrounds, glass UI, click-to-analyze a single player with instant highlights.

📖 What we learned

• Grounding generative features in deterministic geometry (meters, timestamps) yields trustworthy coaching insights.
  
• Tight schema validation + repair loops (zod) turn noisy sketches into robust simulations.
  
• Small, dense summaries make embeddings far more useful for search & retrieval than raw text.

🚀 What's next for PlaySmith

• Production agent stack: Migrate parsing/metrics to Google ADK + Vertex AI Agent Engine for memory, tools, and reliability.
  
• Real Veo/Imagen pipeline: Swap stubs for true video generation + style packs (broadcast, chalkboard, tactical).
  
• xP & outcome modeling: Expected points/expected possession value from distance, angle, and help rotations.
  
• Team library & sharing: Publish, remix, and compare plays across teams; attach tags, counters, and drill scripts.
  
• On-device AR: One-tap AR walkthrough with player cones and ghost trails for practice sessions.

Built With

• gemini
• next.js
• react
• typescript