Over 32 million Americans - almost 1 out of 10 - are illiterate. When we try and register for jobs or services, we always have to fill out forms - but not everyone has the means or the ability to do so, whether it's a language barrier or a physical disability. Our app seeks to fix that problem.
What it does
Comotium, based off of the latin word for "easy", helps people fill out their forms through voice. It scans in a picture of a paper form, finds out where the fields to fill things in are, and then prompts the user for the answers, and fills it all in at the end.
How we built it
We used rev.ai for the real-time speech processing, Microsoft Azure for the backend, Google Cloud for the speech-to-text processing, and built it in Android Studio using flutter.
Challenges we ran into
We had a lot of trouble doing real-time speech processing and knowing when and how to stop listening to the user, and then start re-prompting once we had a new question to ask. Luckily, we were able to get it working!
Accomplishments that we're proud of
We're really happy about the integrations that we had in this project - all of them are integral to how the product works, and we feel like we were really able to get a good use case that help others by combining these.
What we learned
A lot of the technologies used in the project were new to all of us - none of us had any prior experience with flutter, and we also how to work with the (really cool) rev.ai API, and audio streams.
What's next for Comotium
We want to make our form field detection better, save forms so that you can make a central repository of forms that everyone can choose from, and implement the ability to detect corners on a page.