FlowSpeak

Introduction
Welcome
User Flow
Application Advantages
AI Integration
Future Vision
Thank You!

Inspiration

Inspired by the pressing issues of the lack of tools to aid with non-verbal people, Flowspeak was born. Pulled from the ideas and struggles of non-verbal communication, Flowspeak is designed to solve most of the issues in terms of accessibility and ease of use.

What it does

Flowspeak utilizes Picture Exchange Communication System (PECS), an evidence-based and proven method of communication for those who are non-verbal. In this context, non-verbal refers to the inability of communicating using verbal and social cues, and mainly affects those who live with autism, cerebral palsy, down syndrome, and many others. With these cards, an online collection can be created, cataloged, and utilized by the end user. Out of a selection of the collection, possible sentences are formed and processed by Google Gemini, and returned as a set of possible expressions. For example, if a user selects the cards “Mother”, “Park”, and “Ice cream”, a possible sentence would be generated as “Mother, I want to go to the park and eat ice cream”. As a result, the text-to-speech engine takes the sentence and describes it out loud.

How we built it

Our tech stack mainly consists of a traditional full-stack application: React Native for the frontend, FastAPI and SQLite for the backend. Google Gemini powers the possible prompts, which allows for a much faster, and widely accessible platform. The backend also contains most of the data processing, as images are stored internally as their original bytes + hashed w/ SHA-512 to create a unique identifier. This would allow for thousands of images to be stored, and migrations to PostgreSQL can be done very easily through SQLModel.

Challenges we ran into

We found our final team member late, which set us behind schedule. Our planning took too long, and software limitations added further delays. We also struggled with React Native issues, forcing late-night fixes. Despite these setbacks, we learned the importance of careful planning and communication.

Accomplishments that we're proud of

What we are most proud of is how we developed much faster processes for development. Utilizing templates, we were able to create our applications much faster and develop standards on the protocol. For example, the backend is designed with a philosophy of rapid iteration rather than spending time learning new technologies. With this, we were able to spearhead operations and develop something that we are ultimately proud of creating in the first place.

What we learned

We learned quite a bit. Ranging from working with React Native and getting emulators properly set up, to faster implementations for backend code and AI integration. This was really a time to shine despite all of the flaws that had been made. We also learned that careful planning really does matter. Without the planning of every single tool used, the designs, and so much more, this would result in finishing the project much faster and not spending so long spending those countless precious hours on it.

What's next for FlowSpeak

FlowSpeak has major potential for a proper deployment. With the right amount of internal funding, and support, we can entirely deploy this into production. The backend server is entirely designed to be operated with tools such as Docker, and our React Native codebase would need to be polished up in order to be deployed. The next step would be to deploy this to the real world and to get some test results!