UI for the mental math experiment

Chronos

Inspiration

As a university student, I constantly push myself to study smarter and work more efficiently. Yet I've had plenty of days when I spent hours in the library and left wondering how much of that time was actually productive.

That made me think: what if we could understand not just how long we spend on a task, but what happens while we're doing it?

I wanted to explore that question through a practical experiment. Rather than claiming to measure focus directly, I built Chronos to investigate whether patterns in task-completion time and head movement can help distinguish how someone solves a maths problem: mentally, using paper, or with a calculator.

The name Chronos reflects the idea at the heart of the project: behaviour unfolds over time, and a final answer tells only part of the story.

What It Does

Chronos is an interactive maths challenge with webcam-based activity analysis. A participant selects a problem-solving method, calibrates their neutral head position, and works through five maths questions.

While they solve each question, Chronos records:

Gameplay signals: question difficulty, time taken, and incorrect attempts.
Computer-vision signals: face visibility, head-pitch deviation from calibration, movement variability, and the proportion of frames with substantial head tilt.
A live movement timeline: how measured head-pitch deviation changes during the question.

When a correct answer is submitted, two machine-learning classifiers independently predict the solving method:

Model A — Gameplay only: uses task difficulty, completion time, and incorrect attempts.
Model B — Gameplay + Vision: uses the same gameplay signals alongside webcam-derived measurements.

Chronos displays both predictions next to the participant's selected activity, making it easy to see whether the computer-vision features changed the model's decision.

Chronos is an exploratory activity-classification prototype, not a cheating detector. Looking down does not establish that somebody used a calculator, and the app does not determine concentration or productivity directly.

How We Built It

I developed Chronos in Python, using:

Tkinter for the desktop interface and question workflow.
OpenCV for live webcam capture and frame processing.
MediaPipe for face detection, facial landmarks, and head-pose estimation.
Gemini to generate maths questions, with an additional validation step before using them.
pandas to process experiment records.
scikit-learn to train and evaluate two Random Forest classifiers.
Matplotlib / a Tkinter canvas to visualize recorded measurements and the live head-movement timeline.
joblib to save and reload trained models inside the application.

The core pipeline is:

Generate and validate maths questions.
Ask the participant to choose a solving method and calibrate their head position.
Record gameplay and head-pose observations while each question is being solved.
Aggregate observations into one labelled CSV record per completed question.
Train two classifiers on different feature sets, keeping whole sessions together during evaluation.
Reload the saved models and display their predictions after each completed question.

For head-pose measurement, I compare each observed pitch angle to the calibrated reference:

$$ \Delta\theta_t = \left|\theta_t - \theta_{\mathrm{neutral}}\right| $$

I then summarize measurements collected over the question, including mean deviation and variability. This captures movement relative to the participant's starting posture, although it cannot identify what object, if any, they are looking at.

Challenges We Faced

Making computer vision reliable. Early versions displayed a working webcam but saved zero usable samples. I had to debug the interaction between Tkinter's camera-update loop, head calibration, and the recording state before the CSV contained valid measurements.

Separating behaviour from assumptions. Head movement is ambiguous. A participant might look down at paper, a calculator, or simply while thinking. I therefore framed the task as classifying labelled solving methods, rather than inferring cheating or treating a particular pose as proof of an action.

Collecting and evaluating limited data. My initial dataset contained 45 completed questions across nine sessions, balanced across mental, paper, and calculator conditions. Because the sessions were recorded by one person, they do not represent independent participants. To reduce leakage, I evaluated on held-out sessions instead of randomly splitting individual questions from the same session.

Discovering that more features do not always mean better predictions. On an initial held-out evaluation of 15 questions, the gameplay-only model correctly classified 6/15 (40.0%), while the gameplay-and-vision model classified 8/15 (53.3%). However, in a later five-question calculator session, the models produced 4/5 (80%) and 1/5 (20%), respectively. Those small samples are not sufficient to establish reliable accuracy or a consistent improvement from vision, but they clearly illustrate why generalization matters.

What We Learned

Building Chronos taught me how to connect a live computer-vision pipeline to an end-to-end supervised-learning experiment: collecting measurements, engineering numerical features, saving labelled data, choosing comparison models, evaluating on held-out sessions, and bringing the models back into a working application.

More importantly, I learned that an AI system can be technically functional without yet being scientifically reliable. A model's prediction is not the same as evidence of what someone did, and a promising score on one small test can change dramatically on the next. Comparing two models helped me question whether the extra webcam features genuinely contributed value rather than assuming they would.

What's Next

I would like to develop Chronos in several directions:

Collect a larger, consent-based dataset from different participants, then test generalization to people excluded from training.
Improve the experimental design with more consistently matched questions and varied activity order.
Preserve temporal head-movement sequences and test whether patterns over time are more informative than averages alone.
Compare vision-only, gameplay-only, and combined models, with repeated session-level evaluation and better uncertainty reporting.
Explore an opt-in, privacy-conscious learning analytics application that helps students reflect on their study habits without making unsupported judgments about attention or misconduct.

Ultimately, Chronos began with a simple question from long library study sessions: is the time we spend on a task enough to understand how we worked? This prototype is my first attempt to investigate that question with data.