SoloLOG

SideBar View
Profile page View
Profile Page App customizations
Settings pane View
Tables Page View
Notifications Panel View
Dashboard Page View 1
Dashboard Page View 2
Dashboard Page View 3

Inspiration

We have had increased suicides due to undetected mental health illnesses. Basically, many people do not fully understand themselves, their emotions and how factors like location and time affect these emotions. They unknowingly voice out their true feelings but never actually track or realize what they mean.

What it does

Using a user’s phone’s microphone the app running on the background listens to all their daily conversations.
For privacy reasons, other peoples voices and other noises are filtered and only the words of the owner are used for analysis.
Also for privacy reasons, this app has no online server nor does it use any web hosted backend. The business logic runs locally and all data recorded is stored to device's local storage. A user can be sure that their data never reaches any cloud.
From the filtered voice files, we can measure how many times in a day the user experiences the 37 emotions we currently support. From these emotions predict percentage risk of suffering from mental health, predict stress levels, show locations and times that have a negative and positive influence on them.
Another feature we check for is the user’s interpersonal skills. We look at the interrupt patterns, word choice, intonations and other verbal cues in conversation and report how a user may come across to others in conversations.
Another feature we have is tracking of professional communication skills. We monitor for wrong pronunciations and report to the user all the words they mistakenly pronounce and the grammatical errors they make while talking.

How we built it

We decided to build a react front end app that will represent the "mobile app" frontend. It will basically have a similar interface and all the same functionality.
All our business logic will run on local terminal and we will use local files to simulate how our app will only run locally offline and all data will be stored on the phones local storage.
We have a neural-net that determines the emotions from speech.
First, a user turns on recording from the front end side of the app.
Then, the microphone records the user's conversations only and checks for pauses to determine breaks between recordings.
Next, we filter out other voices from the audio file.
Next, we convert the speech to text. (We only do this for our MVP because or neural-net uses text files only for now)
Next, we check for emotions displayed and add metadata (timestamps and location stamps) to the generated output for analysis and append the results to an output file.
Finally, our react app reads the data from the output file and using some functions determine stress levels, mental health illness risks and interpersonal and communication skills etc.

Challenges we ran into

Abrupt switch from Mobile app to web app for our MVP while still maintaining the some process logic we were to use if it was a mobile app.
Many deprecated packages.

Accomplishments that we're proud of

Our app can detect emotions which will hopefully help reduce mental illnesses and suicides.
Our app monitors conversational patterns and gives a user feedback on how they may appear to others.
Our app lets users know locations that stress users or make them happy.
Although we do not intend to distribute any of our data, we feel good knowing it can be used to improve social research and human behavioral research.

What we learned

Full stack development.
Machine learning.
Data privacy and security procedures.

What's next for SoloLOG

We plan to build our iOS and Android apps.
We plan to completely move to only audio file analysis to also track non-textual sounds like sighs, cries, and laughter to better predict emotions.
We plan to finish our notification system that warns for depression and other mental hazards even when a user doesn’t open the app to check their status.
We plan to find a way to store the audio files in small compressed formats for users to listen to and hear how they sound in stressful and rude situations. This is to save on amount of local storage we use.
We plan to find more human characteristics that can be obtained from day to day conversations and generate more helpful output that our users will want to see.