Space Cadets

Let’s build a Twitter-Spaces Recommender System

Comment 1

Inspiration

Let’s build a Twitter-Spaces Recommender System!

I tried out a few different twitter spaces and realized that I really enjoyed the experience.

It's feels like going to a meetup, without having to leave your home.

I could imagine this feature of the platform becoming really popular over time: 1. authors could do public readings, followed by real-time Q&A from across the world 2. companies could do earnings announcements etc.

Lots of really cool use cases...

What it does

I wanted to build a system that would recommend Reputable and Relevant spaces based on the users interests.

I had some requirements for the App:

  1. It must run automatically in the background, and be fully algorithmic! No babysitting, with no manual curation!

  2. The algorithm should be clear and transparent! No Black Boxes!

  3. No External Dashboards / UIs ... it should live on twitter.com!

  4. It must recommend content that is both Reputable and Relevant

  5. It must recommend content from all of Twitter, and not be limited by who the user follows

  6. Recommendations shouldn’t be overwhelming - let's just show the top-5, and update it continuously throughout the day

How I built it

Docker + Python + Sanic (asyncio web-server)

I built my own twitter v2 asyncio client from scratch (including OAUTH2)

Challenges I ran into

Once I found the top spaces that I wanted to recommend, I had no place to show the recommendations...

I decided to save the recommendations as bookmarks, so they wouldn't disturb the user, and they could check them at their leisure

Accomplishments that we're proud of

It works, and the results are pretty good!

What we learned

It was actually pretty easy to build

What's next for Space Cadets

Waiting for an endpoint for the captions generated by the spaces, lots of interesting data analysis there...

Built With

Share this project:

Updates

posted an update —

I received a question regarding the exact fields that were being used for making recommendations. I over-simplified the process and the fields used in the video, as it was already getting too long. But you can find the details here:

https://github.com/vgoklani/twitter_chirp_dev_challenge_2022/blob/main/containers/app_server/src/analysis/spaces_analyzer.py#L249

In addition to what was discussed in the video, the spaces title and topics were also used.

Log in or sign up for Devpost to join the conversation.

posted an update —

I've been getting lots of interesting feedback, so I thought I would share some of that here:

  1. I'm a data scientist, and most of my friends are data scientists too.

  • Which means that the recommendations we saw from the app were all fairly similar...

  • But what's been interesting is that we're getting users for the app who have completely different interests, and what we're seeing is that the app is making recommendations from reputable accounts that we've never seen or heard off. That's really cool...

  1. One of my users pointed out that there isn't a simple way of finding spaces on the twitter.com site when using a desktop browser.... unless you are using the app! :) then they are neatly tucked away inside the bookmarks pane

  2. I built the app in a 48-hr (crazy) sprint, and there were lots of things I wish I had time to do. After the judges finish their review, and the code-freeze has been lifted, I would do the following:

  • I've always been a fan of the McDonalds sign - over N billion served! We are obviously no-where near that number, but it is cool to see how many recommendations the app has made. I see everything inside redis; it would be trivial to surface that metric to a landing page.

  • The app is currently only showing spaces from verified accounts. I would change that criteria to either verified or having more than (say) 50k followers to increase the number of recommendations being made... There aren't enough verified accounts making spaces. I also have to be careful about recommending spaces, as the app is not filtering any content.

Log in or sign up for Devpost to join the conversation.