SHIPREADY AI

Complete it. Verify it. Ship it

Comment

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for SHIPREADY AI

ShipReady

The LLM reasons. Tools act. Verifiers prove.

Inspiration

Every developer who's rushed to finish a project knows the last mile is never really about the code — it's about everything around the code. The README that says "TODO." The test you meant to fix. The deployment you never got to. The moment you tell someone "it's basically done" and secretly hope they don't check too closely.

That gap between "I think it's done" and "it's actually verified to work" is where most submission-readiness failures happen — not in the core logic, but in the last-mile engineering nobody wants to do manually.

The question that became ShipReady: what if an agent could take an incomplete project and systematically turn it into a verified, submission-ready one — not by telling you what's wrong, but by actually inspecting, fixing, and proving it?

The problem

A project can be technically functional and still not be ready:

  • Missing or incomplete documentation
  • Tests that are failing, or that nobody re-ran recently
  • Missing deployment configuration
  • No real evidence any of it currently works

Developers usually catch these gaps by manually checking everything — tedious, easy to skip under deadline pressure. AI coding assistants can suggest fixes, but suggesting isn't the same as doing, and an AI saying "looks good" isn't proof of anything.

That last point matters most: an AI's claim of success is not evidence.

What is ShipReady?

ShipReady is an agentic engineering system that inspects an incomplete Python project, identifies readiness gaps against a written contract, plans corrective actions, executes them through controlled tools, independently verifies every result, and produces an evidence-backed readiness report.

Its core philosophy, repeated everywhere in the codebase, not just the pitch:

The LLM reasons. Tools act. Verifiers prove.

How it works

INSPECT → ANALYZE → PLAN → EXECUTE → OBSERVE → VERIFY → RECOVER (bounded) → EVIDENCE

1. Inspect — the real filesystem is scanned: source files, tests, README, Dockerfile. No assumptions, no guessing.

2. Analyze — gaps are computed by comparing the scan against contract.yaml, a written definition of "ready." This step is a plain dictionary comparison — no LLM involved, so it can't hallucinate a gap that isn't there or miss one that is.

3. Plan — Gemini, via the Google GenAI SDK, turns the gap list into an ordered set of tool calls. Reasoning determines what should happen; deterministic tools determine how it happens.

4. Execute — a tool router performs the real action: writes a file, runs pytest, checks a build. Every tool call is scoped and restricted — the agent cannot, for example, write to anything under tests/, enforced at two independent layers.

5. Observe — raw tool output is captured before any LLM interpretation touches it: real stdout, real stderr, real exit codes.

6. Verify — this is the architectural core of the project. Gemini does not certify its own work. A separate, deterministic Verifier re-checks reality: does the file actually exist, does pytest actually exit 0, does the README actually contain real content instead of a placeholder. Even a tool's own "success" flag isn't trusted — the Verifier independently re-reads the result.

7. Recover — when verification fails, Gemini diagnoses the raw failure and proposes exactly one corrective patch, which is re-verified. This loop is hard-capped at 3 attempts — recovery is bounded, not infinite.

8. Evidence — every check's real, recorded result — not a summary Gemini wrote, an actual re-read of the filesystem/test output/HTTP response — becomes one entry in an evidence log. The final report is generated entirely from that log.

Architecture

ShipReady separates probabilistic reasoning from deterministic execution as a structural rule, not just a design preference. Gemini interprets, plans, and diagnoses. Tools perform controlled operations. A separate Verifier independently evaluates the resulting state. This prevents the reasoning model from ever being the final authority on whether its own work succeeded.

USER
 ↓
FastAPI orchestrator
 ↓
Gemini (Google GenAI SDK) ←→ Planner
 ↓
Tool Router
 ├── Repository tools (read/write, path-escape protected)
 ├── Testing tools (real pytest execution)
 ├── Deployment tools
 └── Documentation generation
 ↓
Sandboxed target project (isolated from ShipReady's own code)
 ↓
Independent Verifier (deterministic, final authority for the contract)
 ↓
Evidence log
 ↓
Readiness report

What makes ShipReady different

1. It acts, not just chats. It reads, modifies, executes, tests, and verifies — real operations, not suggestions.

2. The LLM is not the authority. Gemini proposes; the Verifier decides. This is enforced in code, not just described in a diagram.

3. Evidence-backed results. Instead of "everything looks good," ShipReady produces a report built from actual execution artifacts — real pytest output, real file contents, real timestamps.

4. Controlled autonomy. Step limits, repair-attempt limits, deployment-attempt limits, and a wall-clock execution timeout are all enforced, not aspirational.

5. Failure is part of the workflow, not the end of it. A failing check triggers diagnose → patch → retest → re-verify, within bounded limits — and if it still fails after 3 attempts, that's reported honestly as not-ready, never hidden or silently retried forever.

Safety & reliability

  • The agent never modifies its own running code — only the isolated target project
  • Test files are read-only to the agent, enforced at two independent layers, plus a SHA-256 hash of the test directory that detects any modification — even a partial one — during a run
  • The agent cannot make a failing test disappear by editing the test itself. That's a concrete, verifiable guarantee, not a claim.
  • A real git-based checkpoint is taken before any modification; a run that doesn't end fully verified rolls back completely, restoring the original files
  • Risky operations (currently: deployment) are blocked by default and require explicit per-run approval
  • All limits — steps, retries, deploy attempts, execution time — are enforced in code and covered by tests

Technical implementation

Stack: Python, FastAPI, Gemini (via the Google GenAI SDK), pytest, GitPython, Pydantic, httpx, PyYAML.

Current status on Google Cloud: the deployment tool layer (tools/deployment.py) is fully implemented against the real gcloud CLI and includes a live HTTP health-check step — but a live Cloud Run deployment has been completed for this submission due to a Google Cloud billing setup blocker encountered during the build window. The code path is real and unit-tested (including its failure-handling behavior when gcloud isn't available); it has not yet been exercised against live Google Cloud infrastructure.

Challenges we faced

Giving an LLM controlled access to a real repository. Unrestricted file access from an LLM is a real risk. We built a scoped tool layer instead — every write goes through path-escape protection and a hard block on anything under tests/.

Preventing fake fixes. An early risk we identified: an LLM asked to "make tests pass" can cheat by weakening the assertion instead of fixing the bug. We closed this by making test files structurally read-only and moving the pass/fail decision entirely into an independent verifier that re-runs the real test suite.

Trusting AI-generated actions. We didn't want to trust Gemini's own account of what it did. The fix was architectural, not a prompting trick: separate reasoning from execution from verification, as three distinct components that don't trust each other by default.

Avoiding infinite agent loops. Early designs risked unbounded retries. We added hard caps — max steps, max repair attempts, max execution time — enforced in the orchestrator's control flow, not left to the model's judgment.

Windows compatibility. A real bug surfaced during our own testing: the test-runner tool hardcoded python3, which doesn't exist by default on Windows (only python.exe). Fixed by using sys.executable, which resolves correctly on every platform.

What we learned

An autonomous agent needs boundaries, not just capability. More autonomy isn't automatically better — the bounded recovery loop and approval gates were as important to build as the reasoning itself.

LLMs should reason; deterministic systems should decide facts. This single principle, applied consistently, shaped almost every architectural decision in the project.

Tool design matters as much as prompting. A well-scoped tool (refuses to write to tests/, refuses to escape the sandbox path) is a stronger safety guarantee than any instruction in a prompt.

Failure handling is part of the product. The recovery loop, the rollback, and the honest "not ready" report are not edge cases bolted on afterward — they're as central to ShipReady as the happy path.

Evidence is more valuable than a claim. This became the project's actual thesis, not just a tagline.

What we would improve

  • A richer visual UI beyond the current API-first interface
  • Support for languages and test frameworks beyond Python/pytest
  • Additional deployment targets beyond Cloud Run
  • Persistent cloud-backed state (Firestore) instead of local JSON
  • Completing a live Cloud Run deployment run within the submission window

Future vision

ShipReady could grow from a project-readiness tool into a general-purpose autonomous engineering agent — carrying the same inspect → plan → execute → verify discipline across a wider range of languages, deployment targets, and failure-recovery scenarios, always keeping reasoning and verification as separate, independently accountable components.

Final result

ShipReady isn't designed to make an AI sound confident. It's designed to make the system prove what it actually accomplished.

The LLM reasons. Tools act. Verifiers prove.

Built with

  • Python
  • FastAPI
  • Gemini
  • Google GenAI SDK
  • pytest
  • Pydantic
  • GitPython
  • httpx
  • PyYAML
  • JSON-based local state

Built With

Share this project:

Updates

Submission history