Inspiration

Our goal was to help Parkinson patients, who suffer from poor user experience on mobile devices. User interface design guidelines for smartphone applications for people with Parkinson’s disease

Problems

Natural touch interfaces are common:

  • Accuracy way worse
  • Cumbersome to select text
  • Stated by Patrick Pantel in 2016 (smart select)

What it does

We help Parkinson patients with better user experience on mobile devices. The first iteration includes a smart text selection based on AI. The user has to only tap the word and we expand the selection to the desired text selection and copy it right to the clipboard.

  • Smart Algorithm detects if the selected word is associated with other nearby words

How we built it

We use a Flutter mobile app, which connects to a Azure web app, which again connects to either the Text Analytics API or an Ubuntu VM with the latest Zalando Research Flair SequenceTagger running as a Flask API. The app sends the sentence and the selected position and gets the extended selection as a response.

Challenges I ran into

  • Azure problems with the free plan
  • Bad data for this use case from Microsoft Open Data
  • Defining of deep learning task and finding suitable frameworks / libraries

Accomplishments that I'm proud of

  • Built and end-to-end MVP
  • Usage of Cognitive API and own deep learning API

What I learned

  • The interfaces should be defined better in the beginning even in the agile approach

What's next for Select AI

  • Auto Response (Smart Reply)
  • Complete sentences, while writing
  • Selects the words according to user preference/history

Built With

Share this project:
×

Updates