Inspiration
I was inspired by a piece of software called "SponsorBlock" which is a third party add-on for YouTube which crowdsources time segments in YouTube videos which contain sponsors or advertisements, so that it can automatically remove it. It works so well and so seamlessly that I wanted to create something similar, but not just for YouTube.
What it does
This app is a self-hosted API (along with a simple frontend) that seamlessly removes advertisements from audio files like podcasts and other formats. Users upload their audio file, it generates a timestamped transcript using Azure AI Speech STT model—and then use GPT-4o hosted through AI Foundry to intelligently find advertisement segments within the transcription. Finally, it precisely removes these detected ads from the audio file and the user can download their cleaned, ad-free podcast within moments (usually less than 30s even for longer audio files).
How I built it
I built this service using Python and Flask to power the backend API, with integrations for AI Speech and GPT-4o via network requests, where users can use their own deployments of each models. I used FFmpeg in the Python backend to do precise audio editing based on AI-extracted timestamps. For the frontend, I used Svelte with Tailwind CSS to create a simple, friendly, drag-and-drop interface. Finally, I wrapped everything neatly in Docker with Docker compose to allow easy self-hosting.
How I used GitHub Copilot in VSCode
I used Copilot to significantly speed up my development time when working with the flask backend. I could use the chat functionality with o3-mini and claude 3.7 sonnet models to ask questions on how to implement certain features which made my development much more efficient. I also naturally used the new gpt-4o-preview powered completions model for quick code completions in all parts of the code.
Challenges I ran into
One main challenge of this concept, is that sometimes ads contain information that is valuable to certain people, for example, I would sometimes hear an advertisement of another podcast, give it a listen, and it would turn out to be very interesting. So how can I make sure that people still don't miss out, without having to completely disable this app's functionality. I decided to come up with a system which will return the individual segments to the user through the API after removing them from the audio file so if they want to look through them, they have the option to. Another tweak of this feature is that it supports returning summaries of the segments, to make it even easier to get an idea of what they were, including extracting any links for easier navigation.
What I learned
I learned how to integrate and orchestrate multiple AI tools together to build a practical solution to solve a real-world problem I came across. I also gained valuable experience debugging audio file manipulation edge cases and managing performance to deliver accurate outputs.
What's next for ad segment remover
I hope that people building apps such as for podcasts, audiobooks, radio, etc or others will be able to self-host this, and utilize this functionality to improve their users' listening experience.
Log in or sign up for Devpost to join the conversation.