The inspiration came from our intentions to create a useful web application that uses a state-of-the-art sentence embedding library called InferSent. This NLP library allows us to create a semantically valid vector representation of a tweet that we can use to compare semantic differences between tweets. Using this library, we are able to filter a list of tweets beyond the simple key word matches and find which tweets are semantically similar.
What it does
Using InferSent, we are not only able to identify tweets that are similar but also the users that are tweeting those semantically similar tweets. Using this property, our web application is able to recommend user profiles that a user might be interested in following.
Challenges we ran into
Originally, we want to use this NLP library to identify spoilers. So we engineered a pipeline to retrieve Wikipedia summaries of movies and another pipeline to identify movie titles from tweets. However, in the end, we realized that our semantic similarity module built on InferSent is not accurate enough to identify spoilers. Turns out, it is really difficult to identify semantic difference that arise from a tweet to a Wikipedia summary.
So we ended up pivoting, mid hackathon, and completed an application that recommends users based on the weak semantic similarity identifier.
Accomplishments that we're proud of
We are proud that we were able to use the NLP library that we had set out to use, and we are proud that we built both the frontend and the backend in this short amount of time. Every member of our team contributed with to the code repository, and we were able to collaborate virtually as well. Debugging and merging code virtually, turns out, is very difficult a task.
What we learned
We learned a couple of things from challenges we faced: first is that it is hard to find the perfect use cases for an ml library, and that collaboration can be challenging virtually. However, we also learned from our accomplishments as well. A group of engineering students that have never known each other can come together over a weekend to finish a project in a short amount of time.
What's next for Tweet Friend Finder
Hopefully, we will be able to improve some features on this web app to make it more usable:
- Improve the user recommendation accuracy
- fix UI errors
- Add login features so that the users can view the tweets in their own timeline, instead of trending tweets in LA area.