Inspiration
We were inspired to create this project because our comp sci teacher's father is visually impaired, and thus has trouble navigating outside, so we decided to make our project based on that.
What it does
So far, there is not much to it. It is meant to be a chatbot, or question-answer machine. Currently, it can only classify the intent of the user(who is visually impaired and navigating outdoors), which includes searching for obstacles, directions, info about the location of interest, info about the surroundings, and finding the right path. This is essentially text classification at its current state.
How we built it
We created a corpus of around 110 questions with 5 categories mentioned above. We then preprocessed the data by making it lowercase, removing unnecessary symbols, removing stop words, and lemmatizing. This was all done with NLTK. We then used a bag of words model combined with tfidf weighting using sklearn for feature extraction. Finally, we split the data 80/20 and trained our classification model using a logistic regression with an OAA SVM, also with sklearn.
Challenges we ran into
There was a lot of debugging and trying to transfer data. In fact, we even tried to fine tune a pretrained BERT model for Named entity recognition, but creating a new corpus for fine tuning would have taken too long.
Accomplishments that we're proud of
We were quite surprised that that the intent classification had worked so well, peaking at 91% accuracy.
What we learned
We learned how to create our own datasets, though very rough. We also learned a lot of new machine learning concepts while doing this project such as transfer learning.
What's next for The blind can see?!?!?!?! A navigational tool for the blind.
We plan to figure out entity recognition and entity extraction to provide more specific responses. We even plan on integrating google maps API and very possibly computer vision as well( far future).
Log in or sign up for Devpost to join the conversation.