PlotProof for MATLAB

A Codex-native review loop that catches misleading MATLAB charts, applies controlled repairs, and exports reproducible evidence.

Comment

Inspiration

A scientific chart can be polished, readable, and still be wrong. Rule-based chart recommenders are useful, but they can assign meaning the data does not contain. I wanted a workflow that does more than generate another figure: it should compare alternatives, challenge the highest-ranked choice, and leave evidence for every decision.

What it does

PlotProof reads CSV, Excel, or MAT data and uses a MATLAB plotting engine to render two to five ranked candidates. GPT-5.6 Sol in Codex reviews the actual candidate images against the research goal and scores claim support, legibility, accessibility, honesty, and reproducibility.

The review passes through a strict schema before MATLAB starts. Only five controlled figure repairs are allowed. The model never supplies executable MATLAB. The final package contains PNG and SVG figures, a before-and-after comparison, Markdown/JSON evidence, and a self-contained HTML decision report. A companion manifest records the byte count and SHA-256 digest of every source artifact, so a copied bundle can be checked for tampering.

Demonstrated result

The bundled synthetic dataset has baseline and candidate columns representing two methods. A rule-based ranker puts a confidence band first because the goal mentions uncertainty. GPT-5.6 rejects that option because the columns are not lower and upper bounds, selects the labeled multi-line comparison, and requests safe contrast, typography, legend, and grid repairs.

How I built it

Codex was the primary Build Week development environment. It inspected the pre-existing Agent Skill and release gates, added contract tests, implemented candidate manifests and finalization, ran MATLAB R2025a, inspected real rendered pixels, found and fixed a MATLAB JSON-decoding edge case, and repeatedly ran the full release gate. GPT-5.6 Sol performed the checked semantic review shown in the demo.

After the initial submission, I used Codex and test-driven development to add an adversarial benchmark for the model-to-MATLAB boundary. The valid control is accepted; 14 mutations covering executable-action injection, candidate/model spoofing, field smuggling, invalid scores, contradictory verdicts, oversized findings, and malformed JSON are rejected.

For the final submission pass, I used Codex to add the offline evidence surface with test-driven development. The tests cover HTML injection, path traversal, outside-root symlinks, and same-size file tampering. I then ran the real three-candidate workflow in MATLAB R2025a, checked the report at desktop and mobile widths, and ran the complete release gate with MATLAB.

The Build Week implementation uses MATLAB, Python, Bash, JSON, HTML, and GitHub Actions. The repository documents the pre-event baseline and every post-July-13 addition.

Challenges

The hardest part was keeping visual reasoning powerful without letting model output become executable code. The solution is a fail-closed contract: unknown candidates, models, fields, scores, or actions are rejected in Python, and MATLAB enforces the repair allowlist again.

Accomplishments

  • One-command real-MATLAB candidate, review, repair, and evidence demo
  • Strict validation across five scientific-figure quality dimensions
  • 15/15 reproducible contract checks; all 14 adversarial outputs fail closed
  • Safe repairs with defense in depth in Python and MATLAB
  • Path-safe manifests with no private research material
  • One offline HTML report that embeds candidates, scores, findings, repairs, and provenance
  • SHA-256 integrity verification across 12 real MATLAB evidence artifacts
  • Shell, unit, visual, privacy, provenance, contract, and GitHub Actions gates

What I learned

The best rule score is not necessarily the most honest scientific encoding. Visual review becomes more useful when it must produce structured evidence that can be validated, reproduced, and challenged. A narrow allowlist is also easier to test honestly than a broad promise of safe model-generated code.

What's next

I plan to evaluate the review contract on more research domains and add human acceptance feedback while keeping the model-to-MATLAB boundary strict and inspectable.

Judge test path

Fast path without MATLAB: download build-week-2026.2, extract plotproof-review-evidence-build-week-2026.2.zip, and open review_report.html. The report is self-contained: it needs no API key, JavaScript, network connection, or MATLAB installation.

Reproduce from source:

git clone https://github.com/Kkkakania/matlab-plotting-skill.git
cd matlab-plotting-skill
MATLAB_BIN=/Applications/MATLAB_R2025a.app/bin/matlab ./scripts/run_review_demo.sh --out /tmp/matlab-plot-review-demo
python3 scripts/verify_review_bundle.py --manifest /tmp/matlab-plot-review-demo/review_bundle_manifest.json --root /tmp/matlab-plot-review-demo
./scripts/run_review_contract_benchmark.py --validator scripts/validate_plot_review.py --manifest examples/review/multi_series_manifest.json --review examples/review/multi_series_review.json --out /tmp/plot-review-contract-benchmark

The core review is in PR #42, contract hardening is in PR #43, and the offline evidence bundle is in PR #45. The latest main Quality run is green.

Built With

Share this project:

Updates

posted an update —

Final evidence bundle: open it without MATLAB

For the final pass, I focused on the judge path rather than adding another plotting scheme. The reviewed output used to be spread across images, Markdown, and JSON. It now ends in one self-contained review_report.html that embeds all three MATLAB candidates, the selected result, scores, findings, controlled repairs, before/after evidence, and provenance.

The companion manifest records byte counts and SHA-256 digests for 12 real MATLAB artifacts. An independent verifier fails if a file is missing or changed. The builder also rejects path traversal and outside-root symlinks, escapes review text, and applies a no-script/no-network Content Security Policy.

I ran the complete three-candidate workflow in MATLAB R2025a, checked the report at desktop and mobile widths, and reran the full MATLAB release gate. GitHub Actions is green.

After extracting the release asset, open review_report.html. No MATLAB, API key, JavaScript, or network connection is needed for inspection.

Log in or sign up for Devpost to join the conversation.

posted an update —

Hardening the model-to-MATLAB boundary

After submitting the demo, I added a reproducible adversarial benchmark for the strict review contract. It runs one checked GPT-5.6 control plus 14 hostile mutations covering executable-action injection, candidate/model spoofing, field smuggling, invalid score types and ranges, contradictory verdicts, oversized findings, and malformed JSON.

Result: 15/15 checks passed; every adversarial output failed closed. The benchmark now runs in the release gate. PR #43 passed both Quality checks, and the main-branch Quality run is green.

Log in or sign up for Devpost to join the conversation.