What it does
How I built it
Challenges I ran into
Accomplishments that I'm proud of
What I learned
What's next for CodeSense
What we don't realize is that the messaging apps we take for granted are life-changing tools for those living with autism.
Texting removes the delay involved in processing information for those living with autism. Studies show that they communicate more effectively, and can communicate thoughts that are otherwise misunderstood in oral communication. Texting brings those on the autistic spectrum closer to their families, removes social anxiety and stigma, and improves quality of life.
Here's the problem. A key indicator of autism is impaired abilities to understand emotion. With rapid advancements in deep learning technologies for sentiment analysis, why have we not yet filled in this need?
What it does
Through text-to-speech, ChatSense records up to ten seconds of audio, analyzes the user's tone for emotion, uses Google technologies to enable speech-to-text, and sends a color-coded message accompanied by an emoji to show the user's emotional state. By visualizing the speaker's emotional state, ChatSense allows friends, team members and collaborators to clearly communicate.
How we built it
If a user chooses speech-to-text, the app records up to ten seconds of audio and uses the Google Cloud API to translate speech to text. Otherwise, the user inputs their text message. The text is sent to our server, where Vocaturi determines the message's emotional content and sends this information back to the app. Our emotional algorithm then places the message on an emotional spectrum, matches it to an emoji and sends it to a recipient. We used Android Studio to create the app and PythonAnywhere to host the server.
Accomplishments that we're proud of
We are proud of achieving our goal: creating an app that determines speakers' emotions through vocal and textual analysis. The project contains full integration, cloud technology and appealing graphics. We designed a great idea together, divided everything according to our skills and worked well as a team. Codesense motivated us to learn APIs, programming skills and collaborative coding methods.
What we learned
-- Google Cloud API -- Database Manaement -- Android Studio -- Collaborating with GitHub -- Server Configuration --Virtual environments
Challenges we ran into
Our biggest challenge was integrating all of the different facets of the app so that the API, server, app and database could communicate with each other. We also had difficulties with recording audio and uploading files to the server.
What's next for ChatSense
With a larger database, ChatSense will be able to bring together more users. We are also planning to add gif animations that users can send based on their analyzed emotional state.