We focused on the hackathon's challenge to create an app that assist those with disabilities, and wanted to use some of the newest built in functionality provided with modern browsers.
What it does
This app is designed to assist those who face the challenges of vision and hearing impairment in their day to day lives using a variety of visual filters and live audio transcription across languages.
How we built it
We utilized google cloud platform and webkit to offload the most complex tasks involved in creating this application such as transcribing and translating text, and because of that we were able to use a static site hosted on github pages to accomplish this.
Challenges we ran into
We had surprisingly few issues during this hackathon due to frequent testing and modular design, but we did encounter a major roadblock when initially trying to use the google cloud platform API for translation externally due to a relatively new authentication method being required that has limited documentation. Eventually we decided that utilizing google cloud functions would be a good decision because we would be able to keep using a static website and it would eliminate the need for the complex JSON-based token exchange that would otherwise be required.
Accomplishments that we're proud of
We were able to implement many more features than we initially planned on, and still had time to make the site look well polished.
What we learned
We learned that google recently changed the GCP authentication scheme and that we should look into learning their new process so we can use google apis from external apps in the future.
What's next for AccessAR
As the name might imply, integrating this app into developing augmented reality headsets would allow our users to use this application in a low impact way (hands free and without using video passthrough), which is important for those who have to deal with disabilities on a day to day basis. One feature we considered implementing was to let users tap on a piece of text and have it read out to them, however we opted to add the waypoint functionality with our remaining time to give the app a broader appeal.