ABC Vision

A badminton coaching agent built to decline. Of the 705 reports it produced across 1,117 real production jobs, it withheld a skill level in 42.7% - and can always show you why.

Comment

The problem

A skill rating is a claim about a person. Get it wrong and you have told someone they are worse than they are, in front of their club.

So the interesting engineering question is not how often the vision is right. It is whether the system knows when to stop.

What it does

ABC Vision analyses badminton match video with OpenCV 5 — rally detection, court geometry, shuttle tracking — then has to decide which player on court is the subject before it can say anything about them. When it cannot do that confidently, it withholds the result and shows the evidence instead.

This is a production system, not a prototype. It has run 1,117 real jobs for a badminton shop in Taiwan between 2026-09-06 and 2026-09-17.

Live and checkable while you read this:

The page runs the loop twice. Once on the ordinary case, where identity locks and analysis proceeds. Once on the case where the subject sits out a rally, where it runs out of anchors and asks a human instead. Only the second one branches, and the branch is the part worth watching.

Call protocol_info on the endpoint and it returns the OpenCV version of the process that answered:

"opencv_version": "5.0.0",
"opencv_module": "/usr/local/lib/python3.12/site-packages/cv2/__init__.py"

The OpenCV 5 requirement is the kind of claim a report can assert and a reader cannot check. This submission does not ask to be believed.

OpenCV 5, and why the container is 227 MB

The scan path imports only cv2, numpy and the standard library. No PyTorch, no ONNX Runtime, no MediaPipe. That is what makes real video analysis a 227 MB image instead of a multi-gigabyte one, and it is why the AWS component is cheap enough to leave running for the whole judging period.

Agentic Vision: the loop, and what makes it a loop

Identifying a player is not a classification. It is a search, and a measured visual result decides what gets searched next:

  1. open_identity_session — register detections and appearance signatures
  2. propose_identity — anchor somewhere, propose who the subject is
  3. score_identity_plan — how many rallies does that plan actually cover?
  4. Below the gate? Exclude that anchor and propose again. The second call is the point: a measured result changed which plan gets tried next.
  5. Out of anchors? Compare the plans, explain what separates the two people, and ask the human.

The trace recorded by scripts/demo_agentic_loop.py in the repository: 6 tool calls, 4 decisions, 4 of which had a real alternative, 1 request for human approval. A decision with no alternative did not branch on anything and is not counted as one — the trace reports those separately rather than inflating the number. The browser demo above is a different run of the same tools, and its call panel shows its own count.

Evaluation evidence

Over 1,117 jobs, 705 produced a report:

Outcome Jobs % of reported
Published a skill level 390 55.3%
Declined to publish 301 42.7%
— too little of the player could be tracked 213 30.2%
— no usable rallies 31 4.4%
— camera moved too much 26 3.7%
— other insufficiency 15 2.1%
— player could not be identified 13 1.8%
— serve segmentation not trustworthy 3 0.4%
Other 14 2.0%

390 + 301 + 14 = 705. Every percentage carries both denominators, every time. A percentage without one is a way of being wrong on purpose.

Failure cases, including the one that matters most

The failure we care about most is not in the vision at all.

When the loop exhausts its anchors it asks the player to identify themselves. 110 of those requests were never answered. The loop reached these 110 and asked, which is what it does when it cannot resolve identity on its own. Nobody answered, so the job produced no report and no level. We cannot say what would have happened if someone had.

On length, separately: 68 uploads in this export ran five minutes or longer, 10 of those produced a report at all, and 0 of those yielded a level. That is a denominator of ten, so we report it rather than call it a rule — but it is the opposite of what an earlier version of this page claimed, and the withdrawal is recorded in docs/CORRECTIONS.md.

We also cannot say why each request went unanswered: we did not instrument that. One gap we can name is that throughout this data window none of the app's notification types covered "ready for you to pick". Whether that is what cost us the 110, we do not know.

What we got wrong

docs/CORRECTIONS.md in the repository lists what this project published and then found to be wrong — including three errors the submission made about itself:

  • An earlier draft overstated the MCP protocol version. The SDK exposes two numbers and the higher one is not the protocol version.
  • DNS rebinding protection was assumed to be on and was off: the SDK enables it only for a loopback bind, and every container binds 0.0.0.0.
  • This page claimed five minutes was the length that most often yields a level, when the export shipped in the repository says no upload that long has produced one.

The first two were caught by a conformance checker we built and published (MIT). The third was not caught by any tool — people read the claim against the data, after it had already shipped. That is in the corrections file too, because "our tools catch our mistakes" is a weaker statement than it looks if the worst one got past them.

Responsible operation

The analysis engine is never committed and never leaves the owner's AWS account: a named allowlist of eleven files is staged at build time, and the build refuses to run if anything resembling customer data reaches it — a check that runs twice, the second time inside CodeBuild, because a pushed image cannot be un-pushed.

The security group admits only CloudFront's origin-facing prefix list. There is no SSH and no key pair exists. The decision trace redacts by key name and replaces identifiers with truncated SHA-256 handles, and that redaction is covered by tests rather than by convention.

Built With

Share this project:

Updates

Submission history