Empathetic and Inclusive Design

TidBits is a journaling website where individuals can talk about their moods and feelings. Prior to each journaling session, the web app will take a picture of the author and analyze their mood using emotion detection. Then, as the user talks, their speech is recorded as text. The website will then analyze the text and correlate specific people, places, and things to emotions that the user implicitly suggests through facial expression and diction. Journaling has been proven to have numerous benefits such as helping participants manage anxiety, reduce stress, and identify triggers.

TidBits started with the idea of using AI to design a chatbot app to assist mental health patients by replacing or lowering the cost of therapy. Since we were concerned about the repercussions of giving inaccurate advice, we decided to take a journaling and therapeutic approach to this issue. Inclusive design is also important to us, so we wanted to challenge ourselves and create a design that was accessible to customers who are unable to (or prefer not to) type or write. As we brainstormed and researched, we found that mobility issues affect about 15.5% of Americans, ranging anywhere from loss of fine motor skills to being completely unable to move independently. That's why we added a speech to text feature in contrast to traditional typing or writing. We found that those with mobility impairments struggle not only physically, but emotionally and socially as well. They tend to be socially alienated, anxious, and frustrated. Journaling can have a huge positive impact on these issues, and we hope that by encouraging individuals to verbalize their struggles, they can start a journey of healing.

Building TidBits

TidBits was built using a Flask/Python backend and a HTML/CSS/JavaScript front end. We used Flask to build an API on top of Azure Cognitive Services--specifically the speech recognition and computer vision APIs.

Challenges

Our largest challenges during the creation of this projected stemmed from compatibility, and scope. We were all independently very successful, and were able to implement the piece of code that we were assigned. What came next was a mess of merge conflicts, backups, and refactoring. While we all planned how our pieces of code would work, we didn't spend enough time considering how they would work together, some of our files returned JSON, some returned plain-text, and some wrote to .txt's.

We spent a considerable amount of our night rewriting and refactoring code until it finally clicked. At 8am, after a long and sleepless night, we finally got out full stack working, we could click a button on the page, make a call to our Flask API, send information to Azure Cognitive Services, and see the response reflected on the page.

What We Learned

Planning is key. Even if the scope is small, have a plan going in, know what technologies you want to use and how they work together. It might be tempting to just use what you know best, but when it comes down to creating a well integrated app, every piece of it needs to work as part of the whole. Going forward we'll all have a greater understanding of what it means to plan, organize, divide tasks, and excel as a whole rather than just as individuals.

Share this project:

Updates