Inspiration
A shop launches a promotion on Saturday and gets more customers than usual. Did the promotion work, or was Saturday always going to be busy?
That question drew us to the Brevan Howard track. The same problem shows up in finance when a trading strategy looks good because it ran during favourable market conditions. Inspired by the POPPER research on hypothesis testing, we built a game where players can manufacture a convincing result and see which judge falls for it. POPPER research paper
What it does
Fool the AI puts you in charge of a simulated corner shop. Launch a useless offer before a rush, trigger a local event, or roll a die to bring friends into the shop. After two shop days, two AI judges decide whether your offer worked.
The naive judge compares traffic during the offer with the shop’s historical average. The careful judge estimates how many customers would have come anyway, given the hours, weather and events, then checks how strongly the actual counts support a lift. The game reveals the true offer type alongside both verdicts so you can see who gave it credit.
How we built it
We built the shop simulator and game logic in Python, served them through FastAPI, and made the interface with HTML, CSS and JavaScript. The careful judge fits a Poisson regression on about six months of simulated trading without offers. At the end of each round, it compares the observed customer count with its no-offer forecast and scores the evidence across several possible lift sizes. The simulator keeps the true effect separate so the game can score the judges after they decide.
Challenges we faced and what we learned
The hardest trade-off was catching genuine improvements within a short round while making it difficult for a useless offer to pass. We also tested a judge that learned players’ habits. It picked up more genuine offers, but it was easier to fool, so we kept its predictions as commentary and based the final verdict on the round’s evidence.
Across three simulated player strategies, the careful judge called about 0.3% of useless offers real and detected 89–91% of genuine 20% lifts. When we reduced the genuine lift to 8%, detection fell to 13–20% over the same two-day window. That made the trade-off tangible for us. A small improvement can be real and still need more time to measure.
Disclosure: tools, libraries and AI use Generative AI: We used Claude (Anthropic) and ChatGPT (OpenAI) to help write code, tests, statistics and documentation. All design decisions, testing and results were checked by the team. Libraries and APIs: Python, NumPy, pandas, SciPy, statsmodels, matplotlib, FastAPI, Uvicorn, Pydantic, pytest; DeepSeek API (optional event lookup). Method inspired by the POPPER paper (Huang et al., ICML 2025), whose code we did not use. All code was written during the hackathon.
