FitBet

Accepted Challenge
Homepage of mobile app
Sample AI generated workout plan for a day, tailored for user based on his initial assessment and body analysis
Challange push notification - reading blockchain data from smart contract on aleph zero like workout type, challange id, etc
Into Screen
Finished Workout
Generated Body Analysis
Real-Time Exercises analysis Push-ups Down phase
Real-Time Exercises analysis Push-ups Up phase

Inspiration

Inspiration struck when our team, comprised of fitness enthusiasts and tech aficionados, witnessed the pervasive sedentary lifestyle epidemic gripping society. We envisioned FitBet as a solution to this pressing issue, a platform that seamlessly integrated fitness, technology, and community to motivate individuals towards healthier lives.

What it does

FitBet is a groundbreaking fitness app that combines blockchain technology and machine learning to gamify physical activity. Users can set fitness goals, make bets using Aleph Zero tokens, and challenge others to meet their goals. FitBet tracks users' activity through their phone cameras, accurately counting repetitions using ML models. It fosters accountability, motivation, and social interaction, revolutionizing the way people approach fitness.

How we built it

React Native: We utilized React Native for cross-platform development, enabling us to build the app for both iOS and Android with a single codebase.

Expo: Expo was leveraged for rapid development and access to various development tools and libraries.

Node.js: Node.js served as the backend runtime environment, allowing us to build scalable and efficient server-side logic for handling proxy connections to Gemini AI LLM as well as Substrate Polkadot Smart Contracts caller.

Smart Contracts: We deployed smart contracts to the Aleph Zero chain using Rust Ink! for secure and transparent transaction processing.

Tensorflow: Tensorflow.js was used for machine learning tasks, including real-time exercise analysis and repetition counting using MoveNet body tracking model.

Gemini AI: We incorporated Gemini AI Multimodal LLM for advanced body analysis, to provide personalised training plans, as well as general info about the user's body.

Challenges we ran into

Crafting FitBet was a passionate endeavor, but it wasn't without its challenges. We dedicated significant effort to fine-tuning the workings of Tensorflow.js for real-time exercise analysis. The complexity arises from the extensive calculations executed on the client's device, making optimal performance predominantly supported on newer iPhones.

Our journey involved overcoming obstacles in optimizing ink smart contracts and algorithms tailored for diverse exercises. Additionally, ensuring robust data privacy and security added another layer of complexity. Navigating the intricacies of interacting with contracts, particularly with write methods in our React Native app, proved to be a formidable task. Moreover, the scarcity of documentation on mobile substrate apps further heightened the challenge.

Despite these hurdles, each difficulty only heightened our resolve to deliver a seamless, user-centric experience. FitBet stands as a testament to our commitment to overcoming obstacles and providing an exceptional fitness platform for our users.

Accomplishments that we're proud of

We are incredibly proud of the progress we've made with FitBet, achieving several significant milestones:

1. Deployed and Functional Aleph Zero Contract: We successfully deployed and operationalised a robust Aleph Zero contract, ensuring secure and transparent transaction processing within the FitBet ecosystem.

2. Proof of Concept Mobile App (Demo is deployed to Expo Go): Our proof of concept mobile app is now available publicly online in the Expo Go app, allowing users to experience the core functionalities and features of FitBet firsthand. This demo showcases the potential of the FitBet platform and provides a glimpse into its future capabilities. You can simply get Expo Go on your phone, scan our QR code, and give it a try!

3. TensorFlow Integration: We seamlessly integrated TensorFlow into FitBet, enabling powerful machine learning capabilities for real-time exercise analysis and repetition counting. This integration enhances user experience and accuracy, ensuring precise tracking and monitoring of physical activity.

4. Retrieval Augmented Generation with Gemini AI: Through the integration of Gemini AI, FitBet offers advanced machine learning functionalities. This innovative feature enhances exercise analysis and provides personalised recommendations, further enhancing the effectiveness and engagement of the FitBet platform. What do we exactly, is gathering the user's data, and preferences, with our local database of Real-time exercises, that can be used by LLM such as Gemini AI to build a personalised training plan daily.

What we learned

Ink Smart Contract and PolkadotAPI Integration: Delving into the intricacies of Ink smart contracts and utilising the PolkadotAPI for event listening have deepened our understanding of blockchain technology and its application in decentralised applications.

The Importance of Low-Level Architecture: The challenges of integrating various technologies underscored the necessity of a solid, low-level architectural foundation. This insight has been pivotal in ensuring the scalability and robustness of our app.

The Complexity of Integration: We've learned that integrating disparate technologies, especially in a novel context like combining fitness tracking with blockchain and AI, is both challenging and rewarding. This process has honed our problem-solving and technical skills.

Adaptability and Innovation: Navigating the complexities of developing a cross-platform app that includes real-time AI analysis taught us the importance of adaptability and continuous innovation. It highlighted the need to stay abreast of technological advancements and to be ready to evolve our approach to meet user needs and leverage new opportunities.

What's next for FitBet

As a next step, integrating push notifications into our app is a key priority outlined in our roadmap. This feature will allow users to receive personalised notifications regarding pending exercises tailored by our LLM (personalised). Additionally, leveraging event listeners connected with our smart contracts deployed to the Aleph Zero chain, users will receive notifications about pending bets or challenges from friends.

Currently, our app is developed with React Native and Expo, ensuring compatibility with both iOS and Android platforms. However, due to the heavy computational load handled by TensorFlow for running machine learning models locally on client devices, we initially support only the latest iPhone models. Nevertheless, we plan to explore the optimisation of the app for Android devices in the future.

Expanding our smart contracts will enhance the betting system, potentially incorporating features such as proper account abstraction, custodial model, and ramp payment processors for fiat-to-Aleph Zero conversions. Utilising these would enable easier onboarding on the app and in turn bigger adoption. Additionally, we envision introducing in-app leaderboards to highlight the most active users weekly.

Another major step forward involves integrating more TensorFlow-based real-time exercises, including those with equipment for both home and gym settings. Enhancements to real-time exercise analysis will enable the AI to generate speech commands in real-time, improving user interaction.

Furthermore, additional functionalities such as asynchronous exercise analysis and a daily goal calculator for calorie intake and expenditure, along with a diet plan generator powered by AI, are planned. This AI-driven feature will estimate calorie needs based on textual and image inputs, further enhancing user experience and personalisation.

Aleph Zero testnet contract: https://contracts-ui.substrate.io/contract/5HQq4CdCAtAV8o5AKXdiZTTjjGimWoeYsNoiNr37T3WmuCiL