Post-submission BARRED-Fleet research update: from Agents CLI evaluation to symmetric debate + AST slicing
After submitting BARRED-Fleet, I continued working on the underlying local BARRED research harness rather than treating the hackathon submission as the end of the experiment.
The original submission used the Google Agents CLI evaluation path and established the deployed ADK/Cloud Run system, including the 66.30% token-reduction result and deterministic B-Gate verification.
The next question was: why does the debate itself still fail or waste tokens?
I ran a follow-up 2×2 evaluation comparing:
- unassisted baseline
- Pro-only metacognitive reflection
- symmetric Pro + Con metacognition
- symmetric metacognition + Tree-sitter AST program slicing for the Judge
The first finding was that metacognitive steering needs to be symmetric. Pro-only reflection produced 5 attempt-level logic errors in the asymmetric run, while the symmetric configuration reduced this failure mode and allowed Con to produce structured counter-evidence rather than being forced through an exploit-oriented gate.
The second finding was computational: Judge adjudication was consuming a large fraction of the benchmark budget because it was receiving entire C files. The new Tree-sitter slicer preserves byte-level source anchors and full relevant function scopes while falling back to the raw source when parsing is unsafe.
Across five follow-up replicates, the sliced Judge achieved:
92.0% ± 8.4% accepted-row yield
20,292.5 ± 4,125.1 tokens per accepted row
0 attempt-level logic errors in the sliced series
The paired sliced-vs-unpruned comparison produced a reported p = 0.0349 on this five-replicate evaluation.
This is a post-submission research evaluation, not a replacement of the original hackathon benchmark. The original BARRED-Fleet submission remains the submitted artifact; this update documents how the underlying research continued to evolve during judging.
The next step is to expand beyond the current 10-seed pilot and test the architecture on a larger, independently generated benchmark.
Log in or sign up for Devpost to join the conversation.