PolyBox

Sources
Presentation built with

Inspiration

In September 2026, one attacker used AI agents to launch 105 attacks on online retailers in five days. At least 27 companies were compromised and more than 600,000 credit cards were stolen, at a cost of about $25 per target. Around the same time, researchers found AI agents sending hundreds of thousands of requests at US and Canadian government sites.

Detection isn't the problem. Cloudflare, Microsoft and AWS already flag this kind of traffic. The real problem is what happens next. Every fix (a rate limit, a region block, a CAPTCHA) can also block real customers, and nobody can see that cost before shipping it. So teams hesitate for days or weeks while the attacker keeps going.

And when a fix does ship untested, it can hurt. In 2019, a single new security rule took down about 80% of Cloudflare's traffic worldwide. In 2025, one firewall rule locked CircleCI's customers out for 93 minutes. That gap between detecting an attack and safely fixing it is what inspired Polybox.

What it does

Polybox is a safe test lab for security fixes. It sits between the detection tools a company already uses and the decision to deploy a fix:

Your existing tools detect the attack. We don't replace them.
Machine learning sorts traffic into likely attackers and real customers, with a confidence score.
Generative AI proposes solutions by tuning defenses that already exist: how strict a rate limit should be, who it applies to, over what window.
A deterministic replay runs each candidate against the past week of real traffic, off the live site.
Fixes are ranked by attackers stopped vs. customers blocked, and a human picks what to ship.

The AI does the creative part; the replay does the trusted part. Every result is verifiable, not AI guesswork.

How we built it

We focused on the product design, the business model and the pitch:

Product design: We mapped the pipeline (detection → classification → AI proposals → replay → ranking) and chose to run the replay inside the customer's own cloud. Raw traffic never has to leave their environment; Polybox only learns from privacy-safe patterns and outcomes.
Scoring: Each candidate fix $f$ is replayed over last week's traffic and scored on two numbers, the share of attack traffic it stops, $S(f)$, and the share of real customers it blocks, $B(f)$. Blocking customers is far more costly than letting a little attack traffic through, so we rank by $$\text{score}(f) = S(f) - \lambda \, B(f), \quad \lambda \gg 1$$ and the AI keeps adjusting until it finds the best trade-off. For example, a fix that stops 98% of an attack while blocking only 0.2% of customers.
Business model: We start with mid-market e-commerce and marketplaces. Security engineers use Polybox; the CISO buys it. We sell a subscription plus usage, based on traffic replayed and threats tested.
The pitch: We built an animated deck with Reveal.js and GSAP. The hacker's agent swarm, the wall that takes 14 days to build and then blocks a real customer, the smart barrier, and the live leaderboard replaying a week of traffic are all coded animations. The deck runs as one offline HTML file, so hackathon Wi-Fi can't break it.

Challenges we ran into

Fact-checking under time pressure. Our first research notes had errors, like the wrong country, inflated numbers and a company blamed on a blogger's guess. We verified every statistic against its original source and cut what we couldn't back up.
Three minutes is not a lot. We had a lot of good material. Cutting slides and keeping only the big numbers was harder than writing them.
Not getting too technical. Our first AI explanation used charts and firewall parameters. We replaced it with a simple idea: the AI turns a dial between "too loose" and "too strict" until it finds the sweet spot.
Ground truth. You can't perfectly know which traffic is malicious. We designed around that with confidence scores and labels from existing tools, instead of pretending the labels are perfect.

What we learned

The hard part of security today isn't spotting attacks. It's fixing them without hurting your own customers.
AI is most trustworthy when it proposes and something deterministic verifies.
A great pitch shows the problem before it names the product. We only reveal Polybox once the audience feels the pain.
Real numbers, honestly sourced, beat dramatic claims every time.

What's next

Month 3: MVP replay engine on one firewall, with 3 design partners
Month 6: Paid pilots and integrations with detection vendors
Month 9: Self-serve plan alongside Enterprise
Month 12: Replay inside customer clouds, compliance reports, 10 paying customers

Attackers have AI. Now defenders do too.