JARVIS Health

Inspiration

Imagine a soldier in training, pushing their limits in a high-stakes environment, yet their personal wellbeing goes unnoticed. Over-exhaustion can lead to injuries, but traditional assessments rely on subjective judgment, leaving commanders without real-time insights to prevent risks. While wearable devices typically focus on tracking individual health, we recognized that aggregating data from multiple soldiers, especially in military contexts, is essential. That’s why we created JARVIS Health—a solution that provides commanders with a holistic view of their soldiers' health, using multiple data points to enable safer, more effective training.

What it does

JARVIS Health aggregates real-time heart rate and exertion data from wearable devices using the Terra API, focusing on data from multiple soldiers rather than just individuals. This data is displayed on a dashboard that provides commanders with data-driven insights into soldiers' physical states, enabling more informed decisions. By tracking these vital metrics across a group, the system helps identify early signs of over-exhaustion, optimize training, and reduce injury risk, ensuring soldiers train smarter, not harder. Powered by OpenAI’s LLM, our system also analyzes aggregated data to make actionable recommendations, such as transferring load or suggesting breaks, to ensure that each soldier maintains optimal performance and well-being.

How we built it

We developed JARVIS Health using FlutterFlow, a no-code platform that allowed us to quickly design and deploy a user-friendly dashboard for commanders. The app connects to Firestore via Firebase for real-time data storage, enabling seamless synchronization of heart rate and exertion data across multiple soldiers. The Terra API aggregates wearable data, which is then stored in Firestore for easy access and analysis. We also used OpenAI’s GPT-4o-mini model, served by FastAPI, to analyze the aggregated data and generate actionable recommendations based on the wearable data.

For the hackathon, we embraced an Avengers theme, with each soldier represented by an iconic character to make the app both engaging and memorable.

Challenges we ran into

As first-time users of FlutterFlow, we encountered a learning curve with its unique workflow, especially in handling logic and making API calls. While FlutterFlow enabled rapid development, we found it challenging to structure complex queries due to its support for only one level of subcollections. Additionally, we worked with the Terra API to aggregate wearable data, but its limitations in flexibility required us to get creative with data retrieval and integration. We also had limited access to data from only two types of wearables, which constrained the diversity of our dataset.

Incorporating OpenAI’s GPT-4o-mini to provide real-time recommendations was another challenge. Initially, we struggled to get the concise and actionable responses we needed. We found that the AI wasn't always producing the right level of specificity, so we had to experiment with different approaches to prompt engineering. After refining the prompts, we were able to ensure the responses were brief, clear, and helpful, giving commanders high-level, data-driven recommendations. These challenges provided valuable learning experiences and helped us refine our problem-solving skills as we adapted and built the app effectively.

Accomplishments that we're proud of

We’re really proud of how we took on this project and experimented with a lot of ideas and tools to come up with a solid solution in such a short amount of time. Despite being new to FlutterFlow, Terra API, and Firebase, we were able to quickly learn and use them to build a functional dashboard that tracks heart rate and exertion data. We also integrated OpenAI’s GPT-4o-mini to help provide real-time recommendations based on the data, giving commanders better insights to keep their soldiers safe and performing well.

It was exciting to experiment, adapt, and learn new things on the fly, and we’re really proud of what we were able to build under such tight deadlines. The whole process was a great mix of problem-solving, learning, and seeing our ideas come to life.

What we learned

For all of us, it was our first time using a no-code tool like FlutterFlow, and it was a great introduction to how quickly we could bring our ideas to life without needing to write tons of frontend code. Along with that, we got hands-on experience with Terra API and Firebase, which were new to us but became key parts of our solution.

We learned how to integrate these tools to gather and process data, and the importance of thinking creatively when working with limitations. Working with OpenAI’s GPT-4 was also a highlight. It opened our eyes to how AI can generate real-time, actionable insights from raw data, and we were able to incorporate that into our app to help commanders make better decisions.

Overall, this project taught us how to be resourceful, adapt quickly, and blend new technologies in ways that can have a real-world impact. It was a great learning experience and a lot of fun!

What's next for `JARVIS Health`

Looking ahead, we see significant potential to expand JARVIS Health beyond military training. We plan to tailor the platform for sports teams, where coaches and medical staff can monitor athletes’ exertion and recovery, reducing the risk of injuries during intense training sessions. We also envision first responders using the system to track the physical well-being of paramedics, firefighters, and police officers, ensuring they stay at optimal performance levels during high-stress, physically demanding situations. Additionally, we aim to bring JARVIS Health to healthcare providers focusing on elderly care, allowing for continuous monitoring of vital signs to detect early warning signs of health issues, helping to ensure the safety and comfort of seniors in care facilities or at home. By expanding into these areas, we hope to create a broader impact on health and performance across a range of critical sectors.