Unseen

Begin your day
Travel across campus
Deal with migraines
Get through class

Inspiration

Our team was discussing the additional cognitive load that come with experiences such as chronic illness. Many students, us included, have played a simulation game called Spent, where students are expected to manage their money and make difficult decisions to survive 30 days without being evicted, failing to take care of their families, etc. We wanted to design a software with a similar concept but which could deliver a new message.

What it does

We developed a game which simulates the life of an engineering student with migraines and a mobility impairment. The student is expected to simultaneously take care of their health and maintain their GPA by attending classes, going to lab open hours and studying. Should they neglect their health and allow their "energy" to fall to zero, they will faint and be transported to the hospital, ending the game. Should the student neglect their academics by allowing their GPA to fall below 2.5, their scholarship will be rescinded, also ending the game.

Before players begin playing, a facilitator lists off what classes players are expected to attend, the consequences for skipping class/missing parts of class, where all the tasks are located around campus, what actions players must take every 15-40 seconds to avoid flare-ups, what players should do if they should experience a flare up, and other limitations they may experience (such as paratransit potentially arriving late or not showing up at all).

Then, players are left to figure the game out on their own. Just like in real life, players have complete control over how they spend their time -- there are no multiple choice, pre-established decision trees or similar structures to guide players. Players are free to, at any time, sleep, eat, study, visit the Health Center, leave/return to class, etc., with a complex web of logic ensuring that they feel the results (good and bad) of their decision for the rest of the game. For instance, a player can choose to take paratransit which ends up being late, which then causes them to miss a quiz at the start of a class, which then results in a GPA penalty, which could then cause extra stress which then worsens symptoms.

These sophisticated mechanics were not written by hand; instead, we put our teammate's verbal description of life with chronic illness as well as another teammates' engineering schedule into an AI model, which then "compiled" it into a Python file which kept all the necessary state and did all the necessary calculations to implement our complex rules. We also gave the AI model knowledge of our rendering & real time game management system so that it could insert appropriate API calls into the "compiled" Python file.

How we built it

We started by establishing the experience that we wished to create for our users by discussing our own lived experiences. One of our teammates lives with a chronic illness, which is how we got our current idea.

We then agreed on the delivery format: a mental game which forces users to choose what to sacrifice as well as keep up with a disproportionately large number of health and academic tasks.

Then, we broke off and leveraged our individual strengths. Some of us worked on the system responsible for monitoring of player health, while others drew the art for the project, and others still worked on curating the character and backstory which would allow us to have this demonstration in the first place.

We iterated through various prototypes to experiment with new concepts. For instance, we piloted the "AI compiler" system with a more simple set of rules to see if current AI models would be able to build a program which made sense. We also piloted a command line version of the game to see if it would require the amount of mental engagement we believed it would.

In the end, we combined all of our work and sought feedback from fellow hackers, which we discuss more in the Accomplishments section.

Challenges we ran into

One of our bigger challenges was debugging AI-produced output: when the complex web of logic produced by AI would generate outputs that were contrary to our expectations (i.e. not clearing certain to-do items even though they showed as complete), we found it difficult to troubleshoot. We solved this problem by forcing our AI model to report what game constructs it was using to implement certain rules and check for certain conditions.

Accomplishments that we're proud of

The biggest thing that we are proud of is not the technology we used or the scale of our application. It's the fact that other people engaged with the product inquisitively and reflected on how difficult the experience was. Out of nearly a dozen players, not one was able to survive until bedtime while still having a GPA high enough to keep the scholarship. Almost everyone vocalized how difficult it was to focus on both health and academics when both demand high levels of attention, and many were able to use the game to identify practices which unfairly affect students with chronic illness, such as policies emphasizing perfect attendance.

What we learned

It is very important to agree on the structure of the expected deliverable early on, as well as agree on the interface between different components early on. This includes clearly delineating what each module of a larger project is responsible for and what it is not responsible for.

What's next for Unseen

We want to add more experiences that people with different disabilities and chronic illness may encounter. We also hope to turn this project into a platform for building more online interactives which provoke people into thinking and teach in a way which an article or video cannot.