Inspiration - Elvin
Isaac practices kung fu, Shrey plays squash, and Anthony rock climbs, these are all niche sports. While apps exist for other, more mainstream sports like basketball that use a phone camera to track movement and provide real-time feedback, no such apps exist for many niche sports. This inspired us to build a tool that can create an AI coach and analyse whatever the user might be practicing - squash, kung fu, or anything else that doesn’t have its own app.
What it does - Sebastian
Our project is made for the sports that never get any attention, the “niche” sports. SportsBuilder creates personalized LLMs for your specific niche sport needs. Using advanced motion-tracking technology, we can provide specific, accurate feedback on your form and ability. With our advanced features, we can provide frame-by-frame analysis of each joint's velocity and acceleration, plus the ability to examine specific body parts and their motion. Additionally, the AI coach trained on your video compares your performance to the correct way and provides you with tips to improve, including drills for you to practice.
How we built it - Isaac
As humans, we architected the app, designed the tech stack, and clearly specified the features and UI of the app while Codex implemented the details. At the start, we made a detailed multi-page specification of what we wanted and referenced a design system from another project to improve Codex’s design. After we built an initial app, we engaged in quality assurance en masse. Through a shared Google Doc that Codex reviewed every 30 seconds, we could coordinate debugging.
We ensured that the live AI coach-builder on the app could combine large building blocks. Thus, the coach-builder is reliable and fast as it simply creates new metrics and prompts for the coach (based on conversation with the user). Our code effectively tracks joint movement, velocity, acceleration, and other metrics; and our code ensures the LLM coach can see the video.
Challenges we ran into - Milan
We ran into several challenges with launching the website whilst trying to debug it. Eventually, after about an hour of finagling, we managed to get git, bun, uv, etc all installed on everyone’s computer and the website. Initially, we also struggled to find a way for everyone to contribute given only one Codex instance was actually implementing, so we engaged in quality assurance en masse and used Google Docs for live collaboration
Accomplishments that we're proud of - Shrey
While building this project, we had many points where we demonstrated teamwork, which we were very proud of. One of these moments was when we were all seated around Isaac’s computer, and answering Codex’s questions about the small things about the website that we wanted to implement. All of us were huddled together, discussing our future goals about the project and each of us proposing new features and responding with excitement to each other’s innovations. This moment helped us grow together as a team, especially since we didn’t know each other too well before this hackathon. After this hackathon, we know that we will work together on many more projects. Another moment and accomplishment we are proud of is when we were working together to bug fix and improve the website. Fixing the little things in a project is often viewed as the most boring part, but we genuinely enjoyed doing this together; whether we were laughing at things which went wrong on the website, or asking each other questions like “Is that supposed to be there?”, we learned a lot about bug fixing from Isaac. Additionally, we worked very quickly on this part, typing out our fixes into a Google Doc and Isaac sending those fixes to Codex. We worked like a well-oiled machine, learning from each other and improving the project at the same time. Overall, this project and hackathon have been great experiences and we are all very proud of the project we made.
What we learned - Anthony
We discovered many versatile AI/ML tools—a custom model for joint tracking and LLMs for building—that let us use high-quality image processing and user input processing in our software. This not only improved the user experience for this application, but also opened our minds to valuable possibilities for applying these tools in other aspects of our app. During the ideation process, we gained a deeper appreciation for the importance of empathizing with users to develop a truly useful application that solves a real-world problem of building for niche sports coaches. With the rise of AI coding tools, developers' greatest value has shifted from writing code to developing products to solve problems and help people. Finally, we discovered the limitations of AI coding; while it greatly accelerates code-writing, the underlying choices around website and system tech stack still rely on developers' programming mastery and are very worth learning.
What's next for SportsBuilder - Isaac
Looking forward, we hope to make SportsBuilder an even more personalized and effective way to make coaches for your niche sport. Through personal accounts for every user, we hope to make our coach consider an athlete’s progress over time and learn their history. In addition, we hope to allow athletes to visualize the movements—of themselves or a professional—in 3D and see the minute details of every joint. To make such visualizations, we would use or train a custom 3D model to extract such details from a potentially low-quality video. To improve as an athlete, we will allow athletes to position videos side by side and compare their forms. For example, they could put themselves versus a pro to see how they should improve. Furthermore, we would allow users to share their coaches and fork them, so we would make a community of athletes in niche sports. We would use the $1,000 to handle scaling (e.g. video storage, AI tokens, etc), train additional custom models (e.g. 2D video → 3D joint meshes), and buy more Codex credits to implement these and more features. As players in relatively niche sports ourselves—Isaac practices kungfu, Shrey plays squash, Anthony rock climbs, and our entire school loves hacky sack and spike ball—we will bring SportsBuilder into our communities.
