Inspiration
Most fitness applications present workouts as numbers, charts, and calorie estimates. While these metrics are useful, they often feel too abstract and not very memorable. We wanted to make workout feedback more engaging by turning real body movement into a game-like experience.
Pey-Man was inspired by the idea of combining MATLAB Mobile sensor data with a retro Pac-Man-style interface. Instead of only showing how far a user walked or how many calories were burned, Pey-Man tells a short workout story: what activity was performed, how the body behaved, when fatigue started to appear, and how reliable the sensor-based results are.
What it does
Pey-Man is a MATLAB Mobile fitness tracker that transforms phone sensor recordings into a Pac-Man-inspired workout dashboard.
The system loads accelerometer data and optional GPS data from MATLAB Mobile, processes the workout session, extracts motion features, classifies the user’s activity, calculates fitness metrics, and visualizes the results through a retro arcade-style MATLAB interface.
Pey-Man currently supports activity classification for sitting, walking, and running. It also calculates explainable workout metrics such as workout quality score, fatigue index, confidence index, step count, distance, cadence, estimated calories, active minutes, activity mix, peak fatigue moment, and coach-style advice.
For the bundled example session, Pey-Man detects a walking workout with approximately 624 steps, 0.66 km distance, 26 kcal burned, 59/100 workout quality, 17.2/100 fatigue, and 91.6% confidence.
How we built it
We built Pey-Man entirely around MATLAB and MATLAB Mobile. The workflow starts with phone sensor recordings, mainly accelerometer data and optionally GPS data. The raw acceleration signal is normalized, preprocessed, and divided into sliding time windows.
For each window, Pey-Man extracts features such as dynamic acceleration, RMS value, peak count, active ratio, dominant frequency, spectral power, and optional GPS speed. These features are then passed into an activity classifier.
When the Statistics and Machine Learning Toolbox is available, the project trains a bagged trees ensemble using fitcensemble. To keep the project robust on different MATLAB installations, we also included fallback options such as decision tree classification, centroid-based classification, and rule-based classification.
After classification, Pey-Man calculates workout quality, fatigue, confidence, calorie estimates, activity mix, and other metrics. The project exports machine-readable outputs such as JSON and CSV files, including latest_metrics.json, activity_mix.csv, calories_by_activity.csv, window_features.csv, and fatigue_timeline.csv.
Finally, we built a pure MATLAB pixel-art UI using uifigure. The interface maps real model outputs into arcade elements: workout quality becomes game progress, fatigue becomes ghost pressure, confidence becomes sensor trust, and calories become bonus rewards.
Challenges we ran into
One of the main challenges was making the system reliable with real-world phone sensor data. Accelerometer signals can vary depending on how the phone is held, where it is placed, and how consistently the user moves. Because of this, preprocessing and feature extraction had to be designed carefully.
Another challenge was keeping the machine learning part flexible. Not every MATLAB installation has the same toolbox support, so we needed fallback classifiers to make sure the demo could still run even without the full machine learning toolbox.
Designing the Fatigue Index was also challenging. We did not want it to be a black-box score. Instead, we wanted it to be explainable, using factors such as movement load, movement consistency, cadence drop, and late-session effort.
The final challenge was building an arcade-style user interface using only MATLAB. We avoided external images and JavaScript, so the Pac-Man-inspired visual layer had to be created directly with MATLAB graphics and UI components.
Accomplishments that we're proud of
We are proud that Pey-Man combines real sensor processing, machine learning, fitness analytics, and game-inspired visualization into one complete MATLAB project.
The system is not only a visual demo; it has a full data pipeline from MATLAB Mobile recordings to exported metrics and final UI. It can classify activities, estimate workout performance, calculate fatigue, and generate readable artifacts for analysis.
We are also proud of making the project robust. Pey-Man includes synthetic fallback data, multiple classifier options, exported JSON/CSV files, generated figures, documentation, demo scripts, and a presentation-ready MATLAB pixel-art interface.
Another accomplishment is the explainability of the project. Instead of simply outputting a score, Pey-Man shows how the workout was interpreted and provides confidence and fatigue information to help users understand the result.
What we learned
We learned how powerful MATLAB Mobile can be for collecting real-world sensor data and turning it into a complete signal processing and machine learning workflow.
We also learned that activity recognition is not only about training a classifier. Preprocessing, windowing, feature extraction, sensor quality, and validation are just as important as the machine learning model itself.
During the project, we gained experience in building fallback paths for different MATLAB environments. This helped us understand how to make a hackathon project more portable and easier to demo.
We also learned how to design explainable fitness metrics. A score is more useful when users can understand why it changed and what part of the workout affected it.
Finally, we learned that MATLAB can be used not only for analysis and algorithms, but also for creative interfaces and interactive storytelling.
What's next for PEY_MAN
The next step is to test Pey-Man with more real MATLAB Mobile workout sessions from different users and phone positions. This will help improve the classifier and make the fatigue and confidence metrics more reliable.
We also want to expand the activity classes beyond sitting, walking, and running. Future versions could include cycling, stairs, interval training, and mixed workouts.
Another goal is to improve personalization. Pey-Man could learn a user’s baseline over time and compare each new workout against their own previous performance.
We also plan to polish the Pac-Man-inspired UI with more interactive game elements, better animations, and clearer visual feedback.
In the long term, Pey-Man could become a MATLAB-based fitness analytics platform that turns sensor data into an understandable, explainable, and memorable workout story.
