Equities Filter

Finds US stocks with an unusual burst of social-media and news chatter backed by real news, filters out bots, hype and weak fundamentals, and explains every pick in plain English.

Comment

Inspiration

Every week some stock "goes viral" on StockTwits or in the financial news, and retail investors pile in. Sometimes there is real news behind the buzz. Often it is a few accounts pumping a ticker, bots, or sarcasm that a keyword counter reads as excitement. I wanted a tool that tells those two apart: which US stocks are people suddenly talking about, is there a genuine reason, and can it explain its reasoning in plain English instead of handing out a black-box "buy" list.

What it does

Equities Filter reads the last 7 days of StockTwits posts and financial news, then passes every US-listed stock that was mentioned through a 5-phase screen:

Bot elimination. Drops posts from accounts under 30 days old, with fewer than 3 followers, posting more than 5 times a minute, or copy-pasting the same text across accounts. Hard filters. The chatter must be unusual for that stock (a z-score against its own history), backed by real trading volume (at least 0.8x normal), written by many different people (at least 70% unique authors), and the company must pass basic fundamentals (profitable, growing revenue, debt/equity below 200%). Sentiment. FinBERT plus a trader-slang dictionary that learns new terms ("puts", "dump", "surge") from users' own Bullish/Bearish tags, scoring each ticker separately in posts that mention several. AI review. An LLM checks whether there is a genuine news catalyst and flags hype and sarcasm. Risk and diversification. Overextension, volatility, upcoming earnings and short interest checks, with at most 2 picks per industry. The result is a weekly shortlist on a Streamlit website with price charts, a step-by-step funnel showing how many stocks survived each phase, a methodology page and an honest "how far to trust it" section. An Ask tab lets anyone ask questions like "Why wasn't META picked?". It explains which check dropped the stock, and pulls live Yahoo Finance data and headlines for stocks outside the screen, clearly labelled as such.

How we built it

Data collection: StockTwits API, RSS feeds (CNBC, MarketWatch, Investing.com, Nasdaq) and the Finnhub free tier, stored in SQLite with SQLAlchemy. Screening pipeline: Python and pandas, with yfinance for market data and fundamentals, Hugging Face ProsusAI/finbert for sentiment, spaCy for company-name recognition, and gpt-4o-mini for catalyst and sarcasm review (answers are cached so reruns cost nothing). Frontend: Streamlit and Altair, reading one JSON result file that the pipeline publishes. Automation: GitHub Actions collects chatter three times a day and runs the full screen after every US trading day, then commits the result, which redeploys the site on Streamlit Community Cloud. The database lives on a separate git branch so its history survives between runs. Everything except the LLM is free, and the LLM runs on a $1 budget. So far the whole project has used about $0.07 of it.

Challenges we ran into

Accuracy that lies. About 80% of StockTwits users tag their posts Bullish, so a model that always says "Bullish" looks 80% accurate. I switched to balanced accuracy (the average of the hit rates on Bullish and Bearish posts). FinBERT alone scored about 60%. Adding the learned slang dictionary raised it to about 68%. "Popular" is not "unusual". Apple and Nvidia are always discussed, so raw mention counts are useless. The screen compares each stock's chatter with its own history, which needs several days of snapshots before it works, so it falls back to comparing with other stocks until then. Sarcasm and multi-ticker posts. "Great, another -20% day 🚀" is not bullish. Posts like "$AFRM $KLAR old gags..." need each ticker scored separately. Small bugs with big effects. AT&T was once flagged as a SPAC because its news mentioned "SpaceX". An API key saved with a trailing newline silently broke every Finnhub request in the cloud. Weeks with no picks. Strict rules sometimes leave zero stocks. Instead of loosening them, the app shows the closest runner-ups and explains why each one fell short.

Accomplishments that we're proud of

It runs itself. The site updates every weekday with fresh data, with no laptop involved, for free. It explains every "no", not just every "yes". Ask about any of the 150+ stocks mentioned in a week and it tells you which check dropped it, in plain English. Measurably better sentiment. Teaching FinBERT trader slang raised balanced accuracy from about 60% to about 68% on users' own Bullish/Bearish tags. Honest by design. On weeks when nothing passes, it shows zero picks rather than forcing five, and a "How far to trust it" page lists its limits. Almost free. Only the AI review costs money, and the whole project has used about $0.07 of a $1 budget.

What we learned

How to evaluate a classifier honestly on imbalanced data. How to combine rule-based filters, a domain-specific NLP model and an LLM, each where it is strongest. How to deploy and schedule a full data pipeline for free with GitHub Actions and Streamlit Cloud. That explaining why something was rejected is often more useful to users than the picks themselves.

What's next for Equities Filter

Let it run for months to build a real track record. It already records how each pick performs against the S&P 500, but a few weeks is too short to judge. Add more sources, such as Reddit communities, once a free and reliable feed is available. Email or Telegram alerts when a new screen is published.

Built With

Share this project:

Updates

Submission history