Inspiration

Angel investors usually screen founders alone, without analysts and with little time. AI scoring tools exist, but their scores are hard to interpret: a "70%" rarely tells you how often founders like that actually succeed. I wanted a screening assistant whose numbers mean what they say, so an investor can decide how much to rely on it.

What it does

Halomark is a mobile-first screening tool for angel investors.

  1. Input: share a founder's LinkedIn or website into the app, or type or record what you know.
  2. Research: an agent collects the founder's background as it stood when the company was founded, with a source for every fact, and flags conflicts with your notes.
  3. Questions: the profile is anonymised and checked against 40 yes/no questions about the founder.
  4. Scoring: a model I fine-tuned gives probabilities for five outcome bands (shut down to over $1B) and three timelines.
  5. Verdict: your settings (target outcome, timeline, risk appetite) turn these into Meet, Look closer or Pass. A memo shows the factors behind the verdict, what-if changes, sources and unknowns.

It also supports voice notes, push notifications and screening a whole list of founders from a CSV. It is a screening aid, not investment advice.

How I built it

  • Framework: the pipeline is built on Vela Research's open-source Think–Reason–Learn (TRL) framework:
    • Claude Opus 5.5 wrote the yes/no questions from 2,000 training profiles.
    • The open decision model Kev-0.6B answers them for each founder.
  • Fine-tuning: I fine-tuned three versions of Kev-0.6B on free Kaggle T4 GPUs and calibrated each with temperature scaling. One reads the questions' answers, one reads the raw profile, and the production model predicts outcome and timeline. I also extended TRL with a new step that hands the final decision to a decision model.
  • Data: I built a dataset of 3,000 anonymised founder profiles: 50 based on real founders, 2,950 generated from a seeded model built on those 50. It is split into training, validation and test sets.
  • Research: with the questions and answers held fixed, I compared 8 ways of making the final decision on 600 held-out founders. Every comparison used paired confidence intervals, and the test set was scored only once.
  • App: Claude research agent with web search, FastAPI backend, Next.js web app, ElevenLabs speech-to-text and web push. The models run on my own PC, reachable through a Cloudflare tunnel.

Challenges I ran into

  • No budget. TRL's default answering model has no free tier, so I self-hosted the open Kev-0.6B instead. Kaggle's free T4s have no bf16 support, so everything ran in fp32. I limited the model to the 40 strongest questions so each fine-tune fit in about 3 hours.
  • Silent data loss. Kev's trainer drops any example longer than 384 tokens by default, which removed every example in one condition. My first full run failed late, after training had finished; I found the cause, fixed it and reran.
  • Hosting. Hugging Face Spaces that run code now need a paid plan, so I serve the models from my PC through a tunnel. Cached demo founders are the backup.
  • Format mismatch. My production model was trained with a short description for each answer option. The scoring code sent the options without those descriptions, which shifted predictions by up to 18 points. I caught it by testing the deployed model end to end, then fixed the framework.
  • Realistic data. I had to tune the synthetic data so it was neither trivially easy nor pure noise, and enforce a founding-date cutoff so no later information leaks into a prediction.

What I learned

  • Calibration and accuracy are different things. Fine-tuning cut the model's calibration error from 0.26 to 0.01, the best of every method I tested. It did not beat the simple elastic-net baseline on accuracy.
  • Know the ceiling before chasing it. Because the synthetic data comes from a known rule, I could measure the best possible score. The baseline already reached about 93% of it, so there was little left for any method to find.
  • Calibrated models can still be cautious. The fine-tuned model's probabilities stay close to the base rate. So the app shows the precision and recall to expect at each risk setting, rather than one fixed cut-off.
  • The questions help explain more than predict. A model reading the raw profile did as well as one reading the answers. The questions' value is showing why a founder was flagged.
  • A model must be asked exactly the way it was trained. Small changes to the request can quietly change its answers.

What's next

  • Test on real founder outcomes (VCBench) instead of mostly generated data.
  • Train larger models.
  • Contribute the new decision step back to the TRL framework as a pull request.

Built With

Share this project:

Updates

Submission history