While using Instagram Reels on my computer, I noticed how clunky it felt compared to the mobile experience. Jokingly, I thought: “What if I could just yell at my screen to scroll ” That little spark of humor turned into a real idea: scrolling Reels using voice or sound.
What I Learned
How to use the Web Speech API for voice recognition in a Chrome extension.
How to detect ambient noise levels with the Web Audio API.
How modern web apps like Instagram and TikTok ignore synthetic keypresses for security reasons.
How to work around browser sandboxing by using a local Node.js server to trigger trusted OS-level key events (like real arrow key presses).
5.How desktop experiences differ from mobile, and how to create a funny, yet functional hands-free experience.
How We Built It Chrome Extension We started with a basic extension that listens for voice commands or loud noises using SpeechRecognition and the AudioContext APIs.
Noise Detection Then we measured average decibel levels from the mic input, and once a certain threshold (e.g. clapping or yelling "next") was reached, it triggered a scroll.
Scrolling Logic Originally, we tried to simulate a keydown event for the ArrowDown key. But Instagram and TikTok don’t trust synthetic events, so that approach failed.
Native Bridge (The Hacky Bit) To work around this, we built a tiny Node.js server that runs on my machine and uses AppleScript to simulate actual keypresses (e.g., Down Arrow). This allowed me to bypass browser limitations and actually scroll to the next reel with noise.
Challenges I Faced Browsers don’t allow real keypress simulation from JavaScript — only native apps can.
Instagram/TikTok are heavily sandboxed and ignore most synthetic interactions.
Final Thoughts What started as a joke — yelling at my screen to scroll — became a creative exploration in bridging web and native tech. It might not be the most practical app, but it’s fun and hands-free.
Built With
- i-noticed-how-clunky-it-felt-compared-to-the-mobile-experience.-jokingly
- javascript
- node.js
Log in or sign up for Devpost to join the conversation.