SuSinkt

Hand tracking that determines a scroll up or down

Have you had days where you are just lying on your bed and watching TikTok or YouTube Shorts, but you feel like you don't even have the energy to scroll? Or when you are eating and both your hands are occupied, so you can't even scroll?

This is where SuSinkt comes in. We make scrolling effortless by literally taking it out of your hands. We have created an AI bot that you can tell what kind of content you want to see right now, and it will scroll automatically for you. It even has the ability to learn from your behavior and decide what to show you or what to scroll based on your previous actions.

Features:

You can still manually scroll if the bot skipped or stayed at a YouTube Short, and it will learn from this.
You can scroll and interact using hand and head gestures (This also has the added benefit of making scrolling more accessible physically).

What we learned/Biggest Challenges:

How hard it is to scrap data from Instagram, what an irony lol.
Detecting gestures through camera is really interesting as well as difficult.
Managing consistent pipelines, so different models can talk to each other effectively.

How we built it:

Getting data (Scraping via Selenium, YouTube Data API)
Creating the bot (Sentence Transformers, Gemini API, BLIP)
Automation (Selenium)
Video processing for detecting gestures (Mediapipe, OpenCV)

Future:

We were able to build a working prototype of our bot for YouTube Shorts on web browser, and our next goals would be to take it to the App Store and get it working for TikTok as well as Instagram.