We dream of playing a professional sport but do not have the necessary skills or appropriate guidance. Tailored coaching is often essential for individual players to improve their skills and become successful. The only way to learn and improve is to hire someone with experience to guide them or join a sports academia. Experienced coaches are expensive and sometimes infeasible due to their limited availability. Sports academies find it hard to provide focused attention to individuals. We drop the dream early on and fail to follow the passion of turning our sports enthusiasm into a successful career.

What it does

Pepper - your personal sports companion is a mobile application to collect and analyze player(s) data across matches to suggest a tailored and appropriate training regime to improve their skills and game strategies significantly. The app helps in the overall development of aspiring players by providing them with a professional platform to improve their skills and fitness. We enable connecting them with experienced coaches, premier sports academies and selection committees at various stages of their sports career.

A high-level overview of the solution is as follows:

  • AI-based visual analytics detects and extracts various attributes of the player(s), sports equipment, and environment using a smartphone camera.
  • Player attributes like blood pressure, oxygen levels, step counts, and heartbeat are fetched from their wearable devices.
  • Upon processing, each player is given a performance metric(s). The metric(s) determines the players’ execution of a particular game-skill and their physical fitness throughout the game.
  • The recommendation engine provides appropriate strategies with a set of specific skills and physical exercises to improve upon the weaknesses, such as
    • Exercises that improve the overall strength and stamina of the player
    • Exercises that enable honing game-specific skills
    • Visual feedback to the player for improving overall body posture.
  • The recommendations and players' performance can be accessed from the UI.

AI-modules include Detection, Tracking, Pose-estimation, and Activity recognition are optimized to run in real-time. These modules collect various attributes of the player(s) and their environment, which in turn is utilized to calculate the relevant performance metrics. These metrics and other attributes are processed by the recommendation engine to suggest the appropriate strategies.

How we built it

We are still building the application.

Our approach uses the following building blocks:

  • Object detection techniques like MobileNetv3-SSDLite spatially localize objects of interest in the field of view of the camera.
  • Algorithms such as OpenPose detect various joints in the human body.
  • Specialized tracking algorithms determine variations in poses across frames while performing specific actions.
  • Recommendation engines, based on Reinforcement Learning and Collaborative filtering, recommend game-skills and strength-based exercises to the players.

Detailed feedback is provided on match video recordings, with feedback/suggestions embedded in between specific video slices.

Challenges we ran into

  • Optimizing the technology building blocks to become space and memory efficient, in order to run the application in real-time on a range of low, mid and high-end smartphones and tablets.
  • Contextualizing complex relationships between players and opponents in a game accurately.
  • Defining the strategy to involve diverse profile of players from various sports, age, social-economic backgrounds etc.

Accomplishments that we're proud of

  • The project has the capability to help aspiring people achieve their dreams of becoming a successful sportsperson.

What we learned

  • We learned about the shortcomings and challenges faced by various sports personnel, coaches and sports academies.
  • The range of technologies involved in the project is diverse, enabling us to gain exposure to the entire application development life cycle from its inception stage.

What's next for Pepper: Your personal sports companion

  • Obtain feedback from experienced sportspersons and coaches about the application.
  • Secure funds to scale the project and reach multiple geographical regions.
  • Ensuring that the application follows responsible AI principles to improve fairness, security and privacy.

Built With

Share this project: