SHIP

Teams are shipping features faster than ever, often with high-risk AI vulnerabilities. SHIP is an interactive autonomous gatekeeper that intercepts these vulnerabilities before they hit production.

Comment

Inspiration

Every AI feature your team ships is also a legal or operational decision nobody signed off on. In the current landscape of rapid generative AI integration, fast-moving teams frequently introduce structural compliance hazards: unmasked Social Security numbers inside system prompts, zip codes subtly biasing automated credit underwriting algorithms, or completely unmonitored AI decision loops.

Under the EU AI Act, credit-scoring systems are designated as high-risk, requiring strict human-in-the-loop oversight. Under GDPR, leaking unmasked PII can trigger ruinous financial penalties. Small teams do not have the budget for a dedicated compliance officer, and manual code reviews slow iteration velocities to a crawl. SHIP was inspired by a real, urgent problem: removing the blind spot of AI compliance for fast-moving engineering teams without introducing development friction.

What it does

SHIP is an autonomous compliance gatekeeper that integrates directly into your continuous integration (CI) pipeline. It monitors every pull request the moment it is opened, determines whether a feature has crossed a regulatory line, explains the problem in plain English, cites the rule, and drafts the fix.

Crucially, it shifts the governance model from passive alerts to active enforcement. It works silently in the background on each PR but if a high-risk violation is found, SHIP blocks the merge button, until a person decides to accept the risk or keep the block for code change, with a justification.

For review, instead of forcing users to dig through a messy dashboard, SHIP meets you where you are. You receive alerts on Alexa enabled devices or you can ask status naturally by voice on Alexa ("Alexa, ask ship review what's up."), and the engine automatically coordinates cross-device pushes to stream details to whatever registered display panel you find convenient for you (laptop, TV, tablet or even a smart fridge)

How we built it

SHIP is built from scratch as a highly decoupled, serverless multi-agent pipeline leveraging AWS and the Model Context Protocol (MCP):

  • WORKFLOW ORCHESTRATION: A GitHub webhook triggers an AWS Lambda fast-ack router that parses the incoming pull request. Flagged code fragments are isolated into independent, retryable message blocks dispatched via Amazon SQS to guard against concurrency constraints and allow horizontal worker scaling.
  • CORE AGENT PIPELINE: The analysis loop utilizes the Strands Agents SDK. A specialized Screener executes an abstract syntax tree (AST) pre-filter to catch potential issues at zero cost. Complex anomalies are passed to the Detector Agent which uses Amazon Nova Lite running on Amazon Bedrock AgentCore Runtime, grounded against a local vector corpus of regulatory text via Amazon Bedrock Titan Embeddings.
  • STATE ENFORCEMENT & IDEMPOTENCY: Detector's findings go to Triage, which routes each one: a high-risk finding blocks the merge, and a lower-risk one is flagged for a person to review. Findings are logged as active compliance alerts inside Amazon DynamoDB. Every database operation uses explicit conditional expressions to ensure idempotency across worker retries, so an SQS replay cannot overwrite a human decision.
  • VOICE & DISPLAY RELAY: Powering the voice experience is an independent MCP Server, the same model Alexa+ add-ons use, built on the MCP Python SDK (Spec 2025-11-25) and deployed over a Streamable HTTP Lambda surface with SnapStart enabled. An Alexa Skill carries the conversation to it. The server uses a spec-legitimate, buffered JSON response layer (json_response=True) suited to AWS Lambda Function URL invocation, and a status answer takes about half a second. To bridge stateless serverless execution with live device rendering, we deployed an Amazon API Gateway WebSocket API managed by a dedicated Lambda router, pushing data to the open screen without heavy client polling. Web Push reaches a screen whose page is closed.

Challenges we ran into

  • Stateless Connection Timeouts via Function URLs: The MCP Python SDK answers with a Server-Sent Events (SSE) stream by default (Connection: keep-alive). Behind an AWS Lambda Function URL in its default BUFFERED invocation mode, that stream cannot be carried, and every call failed with a generic timeout after ~240ms, even though invoking the same function directly returned in 375ms. We resolved this by going back to the MCP specification, which explicitly permits a single buffered JSON response over the HTTP transport. Setting json_response=True removed the networking choke point, and a status answer now takes about half a second through the live URL.
  • Alexa Drops Free-Form Answers: Alexa matches every utterance against the skill's sample phrases, even a reply to a question. Our first version asked "What's the reason?" and lost the answer, because "it's only test data" matched no phrase. Now every question is asked through slot elicitation on a catch-all AMAZON.SearchQuery slot, so Alexa hands back whatever the person says, and plain code, not a language model, reads it. A misheard answer cannot become a decision.
  • Two Opposite Decisions That Sound the Same: For free-text slots, Alexa drops the carrier phrase. "Allow the first one" and "block the first one" both reached the skill as "the first one", with no way to tell them apart. For a tool that records approvals, losing the verb is not acceptable. We gave each verb its own intent (AllowIntent, BlockIntent), so the decision can never be inferred wrongly.
  • Reaching Alexa+ From a New Developer Account: The documented CLI path to Alexa+ requires an AWS account that an Alexa Solutions Architect has registered by hand, so a new account gets AccessDenied on the documented role assumption. The track permits a simulated experience, so we built a web page that acts as a full MCP client (initialize, tools/list, tools/call) against our deployed server. Anything that speaks MCP gets the same rules.
  • One Process, Two Requests: On a warm Lambda with SnapStart, the MCP SDK's session manager runs its lifespan only once per process, so the first request succeeded and the second failed. We build the ASGI app inside the handler on every invocation, and reported the gap upstream to the MCP Python SDK project with a proposed docs fix.

Accomplishments that we're proud of

  • 9 for 9 Under Adversarial Testing: We tested SHIP on a pull request against a fork of a real open-source finance app (MicroPyramid/micro-finance), with violations planted among deliberate look-alikes. SHIP caught all 5 violations, cited the correct rule for each, and correctly dismissed all 4 look-alikes.
  • Choosing the Model by Measurement: We benchmarked six models, including Claude Haiku 4.5, Nova Pro, Qwen3, GLM-5 and DeepSeek, on the same planted violations and look-alikes. All six got every case right, so accuracy didn't decide it; speed and capacity did. Amazon Nova Lite answered fastest, at a median of 3.1 seconds against 13.4 for Claude Haiku 4.5, and its limit of 200 requests per minute is 20 times higher, which caps how many pull requests SHIP can review. Same accuracy, faster, with room to grow.
  • Production-Grade Serverless Architecture: SHIP runs entirely on scale-to-zero, pay-per-use serverless infrastructure. It incurs near-zero cost while sitting idle, and it scales out on its own when a large pull request arrives, with each risky fragment reviewed as its own job.
  • Real Pipeline Blocking: We tied the agent's evaluations to GitHub branch protection rules. The tool goes beyond advice; it physically guards the code repository.
  • Human Oversight the Model Cannot Talk Around: The key rules live in the server, not in a prompt. A voice decision is refused unless the finding was reviewed on a screen in the last ten minutes and a reason was given, so nobody can approve something they haven't seen.
  • Free flowing conversation, Not hardcoded commands: You say "laptop", "first one is fine", "it's only test data", and SHIP follows. Voice carries the headline and the decision, and the screen carries the detail.

What we learned

  • Test the Model on Your Own Task: Benchmarking against your own task pays off. On our compliance cases, Amazon Nova Lite matched the larger models, so accuracy didn't separate them. What did was throughput. Bedrock's request quota (200 per minute for Nova Lite against 10 for Claude on my account), not tokens, is what limits how many pull requests SHIP can review.
  • Serverless Means Any Request Can Be a Replay: Running MCP on stateless Lambda taught us to keep state in DynamoDB, not in a session. What a screen showed is keyed on the finding, because a server built fresh on every call has no session to remember. Queues teach the same lesson from the other side. SQS can deliver a message twice, so every write is conditional (attribute_not_exists) instead of a raw PutItem, and a replay can never overwrite a human decision.
  • Silence Must Never Look Safe: A review that hasn't finished can't look like a pass. The ship/compliance check stays pending until every piece of code has reported, and a finding a person blocked keeps blocking until the code changes.
  • Voice for the Headline, Screens for the Detail: Voice is the natural way to hear "7 blockers across 2 pull requests" and to make a decision. A screen is the natural place to read the details. Each does the job it's good at.
  • A Person Stays in Charge Only If the System Makes It the Easy Path: Oversight that depends on good intentions fails. SHIP puts the person in the decision and refuses the decision without a look and a reason.

What's next for SHIP

  • Autonomous Remediation Commits: SHIP already drafts a code patch for every finding. The next step is wiring the GitHub API write-back so that, once a person approves by voice or on screen, the patch is committed to the pull request branch automatically. This may be an optional feature because each organization has a different risk appetite.
  • Learning Across Repositories and Time: SHIP already turns the findings reviewers allow into patterns that give each new review more context. Next, we run that miner on a schedule and extend it across an entire organization's repositories, surfacing compliance trends over time, such as which rules teams accept most often and where.
  • Alexa+ and Fire TV: Bring SHIP to Alexa+ as an add-on, and make Fire TV a native screen for the findings.

Built With

  • alexa+
  • amazon-bedrock
  • amazon-dynamodb
  • amazon-nova-lite
  • amazon-sqs
  • api-gateway-websockets
  • aws-lambda
  • bedrock-agentcore
  • fastapi
  • mcp-server
  • model-context-protocol
  • python
  • strands-sdk
  • titan-embeddings
Share this project:

Updates

Submission history