Inspiration

My friend recently obtained his driver's license, but the struggle to secure a North Carolina DMV appointment was real. Despite daily checks, he couldn't get a slot and even endured a 5-hour wait for a road test. Frustrated by this, we decided to create a web scraping bot that notifies users about new appointments, helping them save time and avoid the hassle.

What it does

Our solution consists of two key components: a Selenium web scraper and a user-friendly Telegram bot interface.

How we built it

For the bot and scraper, we employed Python. We utilized Selenium Grid with Docker for scalability. PostgreSQL was our database of choice. Given the single-page structure of the DMV website, web scraping with Beautiful Soup or finding a suitable API was impossible. Our answer was Selenium. For the Telegram bot, we used the Python-telegram-bot package, leveraging multiple Telegram bot APIs to provide a seamless user experience.

Challenges we ran into

Navigating Telegram bot API documentation was a bit daunting, as we weren't very familiar with it. Web scraping with Selenium presented challenges due to its inherent slowness. We tackled this issue by running concurrent processes, deploying multiple Selenium nodes to scrape the website. This solution can be easily scaled horizontally. Additionally, Telegram bot API imposes message sending limits, prompting us to dynamically break down messages into batches to address the issue.

Accomplishments that we're proud of

We take pride in our aesthetically pleasing website, meticulously designed by Mengjie. We're also thrilled with the Telegram interface we developed, delivering a native app-like experience on users' phones.

What we learned

This project taught us how to effectively use the Telegram bot API and how to scale web scraping solutions using Selenium.

What's next for NC DMV Bot

Our plans for the NC DMV Bot include transforming it into a software-as-a-service (SaaS). We aim to introduce advanced features, such as automatically scheduling appointments on behalf of users or allowing them to specify their preferred times, with the bot securing appointments when available.

Built With

Share this project:

Updates