VocalReflections

"Home" Page
"Entries" Page
"Summary" Page

Inspiration

Amid the social isolation and economic challenges brought on by the COVID-19 pandemic, mental health has emerged as a widespread concern. Research shows that patients struggling with mental health who kept a journal about their thoughts, experienced fewer physical symptoms than patients who did not journal. Journaling allows us to express emotions, reflect on problems, and acknowledge moments of gratitude. This sparked our idea of integrating technology with sentiment analysis to create a speech-to-text journaling app.

What it does

VocalReflections is a speech-to-text journaling app designed to address the prevalent mental health challenges intensified by the COVID-19 pandemic. Through the integration of cutting-edge technology and sentiment analysis, the app empowers users to express their thoughts verbally, fostering emotional well-being and self-reflection.

The "Record Now" feature allows users to effortlessly capture their thoughts by speaking into the app. The client captures and streams audio, converting it into bytes for efficient transmission to the backend Django project.

The "Entries" page serves as a digital journal, displaying entries with details such as date, sentiment (positive, negative, neutral), and the transcribed content generated using Google's speech-to-text API. The sentiment analysis is calculated based on the presence of sentiment keywords in each entry, providing users with insights into their emotional patterns over time.

The "Summary" page offers a comprehensive overview of the user's emotional journey. It calculates the user's average mood, highlighting trends in positivity and negativity. Users can easily identify their most positive and negative entries, facilitating self-awareness and reflection.

How we built it

When the user initiates the recording function, the client begins capturing and streaming audio, converting it into byte form. After stopping the recording, the bytes are transmitted to the backend Django project for further processing. In the backend, the text undergoes conversion to written form using Google's speech-to-text API. Subsequently, the text is forwarded to Google's natural language processing API for sentiment analysis, providing quantitative tone indicators.

To account for potential future expansions, such as a playback feature, and to ensure recovery in case of system crashes, a copy of the audio is stored server-side. The processed text, sentiment analysis results, creation date, and user ID are all saved into a SQL server. Following a successful completion, the backend notifies the frontend, prompting a redirection of the user to the entries page.

Upon loading the entries page, the frontend accesses a backend API to retrieve the list of historical entries, seamlessly integrating them into the page for user access.

Challenges we ran into

The primary hurdle we encountered revolved around managing audio files and various audio types. The challenge arose from the necessity to serialise non-textual data into raw bytes for transmission through a request body. Simultaneously, we had to process these bytes and dispatch a copy of the data structure/schema to the backend. This process introduced complexities as it required meticulous synchronisation of multiple data types across two distinct API layers.

Another significant obstacle was navigating the intricacies of working with Google APIs. The most formidable roadblock was aligning with Google's intricate schema, leading to a concentration of debugging efforts.

Accomplishments that we're proud of

We are proud of overcoming the challenges associated with Google's APIs, our team successfully integrated both the speech-to-text and natural language processing APIs into our application. In addition, tackling the complexities of handling various audio file types and efficiently serializing non-textual data demonstrated our problem-solving skills. We successfully synchronized multiple data types across different API layers, ensuring communication between the frontend and backend.

What we learned

Clear and open communication played a pivotal role in our project's success. We learned that maintaining transparency and fostering an environment where team members can freely express ideas and concerns contributes significantly to overcoming challenges.

The challenges we faced required collaborative problem-solving. Our diverse skill sets allowed us to approach issues from different perspectives, leading to creative solutions. This experience emphasized the value of teamwork in tackling complex problems.

What's next for VocalReflections

Future plans for VocalReflections involve incorporating post-editing and removal functionality, as well as working on a more friendly user-interface.

In addition, potential features include location tags to identify the positivity/negativity of entries from specific places, filtering by different time frames and enabling search and sorting options for posts.