ScoutBot

• 

Inspiration

Every week, hundreds of scholarships, fellowships, and internships open for Nigerian students on a national and international scale, with only a few being publicized and taken advantage of -- and most of them expire unfound.

I built ScoutBot because close friends and coursemates of mine have missed brilliant opportunities like the Afara Initiative, Microsoft internships, and more. Not because they were not qualified. Because they found out too late -- days after the deadline, weeks after it was buried in a blog post they stumbled on by accident. That was the moment I realized the problem was not the opportunities. We are always told at events that Africa needs more opportunities, and we do, but the bigger, quieter problem is distribution.

Nigeria has over 2 million university students. The average student checks 6 to 8 different websites, social media groups, and WhatsApp forwards to stay updated. By the time an opportunity surfaces in their feed, it is often already crowded or closed. I wanted to build something that wakes up every week, does the searching for them, and delivers only the real ones -- clean, current, and directly to their inbox.

What It Does

ScoutBot is an automated opportunity intelligence platform for Nigerian university students. It runs every Sunday at 7 AM WAT, completely on its own, with no human in the loop.

Here is what happens in each cycle:

Scrapes 15+ opportunity platforms simultaneously -- scholarship databases, fellowship portals, internship boards, bootcamp listings, and grant programs, covering both Nigeria-specific and international opportunities open to Nigerians.

Filters and deduplicates -- listings are checked against what is already in the database. Duplicates are dropped before they ever reach a subscriber.

Syncs to a live Google Sheet -- surviving opportunities land in a clean two-tab database (Nigeria / International) with Title, Category, Application Link, Deadline, and Date Added -- accessible to anyone with the link.

Auto-removes expired entries -- any listing whose deadline has passed or that has exceeded the retention window is removed automatically. Subscribers never see a dead link.

Sends a weekly email digest -- a clean, responsive HTML newsletter goes out every Sunday to 507 verified subscribers, grouped by region and category, with one-click Apply links. The system includes IMAP bounce tracking: permanently rejected addresses are blocked from all future sends.

The result is a fully automated pipeline that turns fragmented web noise into a curated, dead-link-free opportunity feed -- delivered every Sunday morning at zero operating cost.

How We Built It

The architecture is intentionally lean. Every design decision started with one question: will this still run correctly in six months when I am not looking at it?

Scraping layer -- Python 15+ site-specific scrapers run using BeautifulSoup4 and Requests. Each scraper handles its source's structure, filters by keyword and recency, and passes only credible, current listings downstream. Items are validated before they ever touch the database.

Storage layer -- Google Sheets + gspread Surviving opportunities are written to a two-tab Google Sheet. Schema migration is automatic -- if the bot detects an old column structure, it resets headers on the next run without manual input. cleanup.py runs after every scrape and removes expired rows using header-name column lookup, making it robust to future schema changes.

Delivery layer -- Gmail SMTP and IMAP notify.py builds a responsive HTML email, assembles the subscriber list from multiple sources, deduplicates and validates every address, records bounces in a persistent Bounced tab in Google Sheets, and sends via Gmail SMTP. One bad address does not abort the whole send -- each address is handled independently.

Scheduling -- cloud-hosted Python scheduler The full pipeline fires every Sunday at 7 AM WAT via a cloud-hosted Python scheduler. No VPS, no paid server, no manual trigger. It runs indefinitely at zero cost.

Community distribution on top Contributors extended the platform with a WhatsApp broadcast bridge built in Node.js using whatsapp-web.js, SQLite, and Express, and a Telegram digest bot. ScoutBot now reaches students across three channels -- email, WhatsApp, and Telegram.

Tech stack: Python 3.11, BeautifulSoup4, Requests, gspread, Google Sheets API, Gmail SMTP and IMAP, python-schedule, Node.js, whatsapp-web.js, SQLite.

Challenges We Ran Into

Cloudflare blocking Major opportunity sites block scraping from cloud server IPs even with real browser user agents. The fix was building direct site-specific scrapers with parser logic tailored to each source, avoiding the blocked paths entirely.

Subscriber email validation Early digests bounced against corporate and university mail servers. I built a validation pipeline: regex format checking, MX record lookup via dnspython, a persistent Bounced tab in Google Sheets blocking resends to known-bad addresses, and per-address error handling so one failure does not stop the rest.

Building for longevity on zero budget Every component had to answer: what happens when I cannot afford to fix this -- when the laptop breaks, the internet drops, or I am deep in exams? That constraint produced the auto-migration logic, header-name column lookups, the 23-day retention cap, and persistent bounce tracking. The system has to take care of itself.

Knowing when to cut I prototyped an AI scoring layer using Google Gemini Flash. Each opportunity was rated 1 to 10 for relevance to Nigerian students. Technically, it worked. On a free-tier quota, it became the most fragile part of the system and the first thing to fail under load. I cut it. The lesson: AI adds real value when the problem is genuinely ambiguous. When budget is tight and reliability is the product, simplicity wins. AI is the next chapter, not the foundation.

Accomplishments We Are Proud Of

507 verified subscribers, grown entirely by word of mouth. Zero paid marketing. Zero advertising budget.

A three-channel distribution system -- email, WhatsApp, Telegram -- built in stages. The core by one person. The extensions by contributors who believed in the project enough to open pull requests.

Zero operating cost. As a student from a middle-class background in a lower-income country, this is the accomplishment I am most proud of. To build is one thing. To build something that sustains itself when you cannot always afford to sustain it yourself is another.

What We Learned

When to leave AI out. Gemini was the most technically interesting part of the project and the first thing cut. AI layers earn their place when the problem is genuinely ambiguous. When reliability is the product, complexity is a liability.

Build for the moment you are not there. Every decision -- auto-migration, bounce tracking, the 23-day cleanup window -- was made by asking: what happens when I cannot fix this? That mindset is what separates a project from a product.

What Is Next for ScoutBot

AI-powered opportunity matching -- students fill a short profile (course, year, interests, GPA) and the system ranks which opportunities are most relevant to them personally. The data is already there; the matching layer is next.

University partnerships -- licensing the digest to student unions and career offices across Nigerian universities.

Mobile app -- push notifications the moment a high-match opportunity drops, before it is crowded.

Pan-African expansion -- Ghana, Kenya, South Africa, and diaspora scholarship databases.

The market is 2 million Nigerian university students. The infrastructure is already built, already running, already serving real people. What is next is scale.

Every week, hundreds of scholarships, fellowships, and internships open for Nigerian students -- and most of them still expire unfound. ScoutBot is changing that, one digest at a time

Built With

• python
• scarpy