Twitter helps users find people who they may be interested in following based on their following/followers network. We believe that this recommendation system, although proven useful, has some room for improvement. Our team came to the conclusion that besides getting recommendations based on account following patterns, it would be nice to get recommendations of people who tweet messages similar to you. This can result in a more precise recommendation system, growth in account exploration, and building a community of users with similar interests.
What it does
Our API helps users find new connections in Twitter by comparing the tweet content of any two Twitter accounts, and displaying any similar tweeting topics found on their timeline.
How we built it
We build this app using Flask and Python for data processing with the help of some NLP libraries, and React for the front-end.
Challenges we ran into
We had some trouble processing and analyzing the tweets from users. We wanted to train a model that would be able to identify the topics present in different tweets using NLP, but we didn't have the time or labeled data available to train said model. Instead, we opted to create a different algorithm that still uses some aspects of the NLP process such as tokenization and lemmatization, but added our own twist that allows for tweet categorization for some initial topics. During this process we also had to develop some heuristics to determine if two twitter accounts tweet messages with similar content.
Accomplishments that we're proud of
We are first of all proud of having a working MVP for this Hackathon. We are also proud in our algorithm that finds the Twitter account similarity without the aid of a trained model. We are aware that a trained model could have better results, but we would need more time and data to train it at a good enough level.
What we learned
One of our biggest takeaways from this hackathon was discovering how powerful the Twitter API is. After working with it for the first time, we were surprised by the great documentation available for developers. We also developed data processing skills and learned how to use some libraries such as NLP. Our participation in this #Codechella leaves us very inspired and satisfied with the progress achieved in one weekend. We are looking forward at using the Twitter API in future projects and continue developing new skills.
What's next for #Interests
Gather a big enough dataset to train a model that is able to identify more categories.