• 

  Python Code Review Support

• 

  Java Code Review Support

• 

  Mulesoft Code Review Support

PolyDev Coach

Inspiration

Every enterprise developer has lived this moment: a production incident traced back to a hard-coded credential that slipped through code review. Or a MuleSoft integration silently swallowing exceptions for months because the right reviewer never saw it. Manual code review does not scale — and for specialised stacks like MuleSoft, finding reviewers who deeply understand integration patterns, APIkit configuration, DataWeave usage, and Mule 4 error handling is genuinely difficult.

This project was inspired by a real pain point. We already had a strong MuleSoft static analysis package that could reliably detect problems — but it was completely silent on why those issues mattered and how to fix them. That led to a simple question:

What if we combined deterministic static analysis with Amazon Nova’s reasoning capabilities?

Not a single chatbot. Not a generic “review my code” prompt.
Instead, a coordinated pipeline of specialist agents — each focused on one responsibility, each routed to the right Amazon Nova model tier for the task.

That is how PolyDev Coach was born.

---

What it does

PolyDev Coach is a multi-agent AI code review platform for MuleSoft, Python, and Java.

Users can:

• Paste source code directly into the UI
• Review Python, Java, or MuleSoft snippets
• (Architecture-ready) upload a MuleSoft project ZIP for full project analysis

PolyDev Coach then runs a 6-agent pipeline that combines deterministic static analysis with Amazon Nova-powered reasoning.

1) Static Analyzer (deterministic foundation)

This is the zero-hallucination layer.

• MuleSoft: uses the open-source mulesoft_package_validator
• Python: uses AST-based checks + lint-style rules
• Java: uses a custom rules engine for common security and maintainability issues

This layer finds concrete issues such as:

• hardcoded credentials
• weak exception handling
• dangerous anti-patterns
• structural maintainability issues

---

2) AI Analyzer (Amazon Nova Micro)

Takes the static findings and enriches them with:

• architectural context
• issue grouping
• higher-level reasoning about code patterns

Rather than asking AI to discover issues from scratch, it reasons over a grounded finding set.

---

3) Coach Agent (Amazon Nova Lite)

For each issue, the coach explains:

• why it matters in production
• what risk it introduces
• how a developer should think about fixing it

This turns code review into developer education, not just issue detection.

---

4) Refactor Agent (Amazon Nova Pro)

Generates a production-ready refactor that:

• preserves business logic
• removes critical risks
• improves reliability and maintainability
• applies safer engineering patterns

---

5) Validator Agent (Amazon Nova Lite)

A separate agent independently evaluates the output and produces:

• a quality/confidence score
• structured validation feedback

If the quality score is below the threshold, the system can automatically retry the refactor with validator feedback as context.

---

6) Optimizer Agent (Amazon Nova Micro)

Final cleanup pass that:

• removes redundancy
• improves consistency
• prioritises critical findings
• normalises structure for a clean user experience

---

Final Output

The user gets:

• severity-coded findings
• production-oriented coaching
• safer refactored code
• before/after comparison
• confidence score from an independent validator

This creates a code review experience that is:

• explainable
• educational
• cost-aware
• self-validating
• enterprise-friendly

---

How we built it

Core architectural idea: route each task to the right Amazon Nova model

Instead of using one model for everything, we designed task-specific model routing.

Amazon Nova routing strategy

• Amazon Nova Micro → structured JSON enrichment and lightweight optimisation
• Amazon Nova Lite → coaching, reasoning, and independent validation
• Amazon Nova Pro → high-quality code generation and refactoring

This matters because each step has a different requirement:

• some steps need speed and low cost
• some need grounded reasoning
• some need stronger code generation quality

That makes PolyDev Coach a true multi-agent architecture, not just multiple prompts calling the same model.

---

Self-correcting pipeline

One of the most important design decisions was introducing an independent validator.

Instead of trusting the refactor blindly:

1. The refactor is generated
   
2. A separate validator reviews it
   
3. The validator scores the result
   
4. If quality is too low, the refactor can be retried with validator feedback
   

This creates a lightweight self-correcting AI pipeline:

• more reliable than a single-pass LLM workflow
• more explainable
• more aligned with enterprise trust expectations

---

Tech stack

• Amazon Bedrock Converse API — unified access to Amazon Nova models
• Amazon Nova (Micro, Lite, Pro) — model routing by task
• FastAPI (Python 3.11) — async backend, typed with Pydantic
• React 18 + Vite + TailwindCSS — frontend UI
• Custom multi-agent orchestration layer — sequential pipeline with structured outputs
• mulesoft_package_validator — open-source MuleSoft static analysis package (published on PyPI)
• AWS App Runner — backend hosting
• Amazon S3 + CloudFront — frontend hosting

Note: The current hackathon build focuses on the live multi-agent pipeline and core review flow. The architecture is ready for richer document-grounding and broader repository integrations as the product evolves.

---

Challenges we ran into

1) Reliable structured outputs from LLMs

Even when asked for JSON, model outputs can still include:

• markdown code fences
• trailing text
• malformed JSON

We solved this by building a strict parsing layer that:

• strips markdown fences
• validates structured output
• catches parsing failures safely
• returns graceful fallbacks instead of breaking the pipeline

---

2) Balancing static analysis with AI

Early versions sent raw code directly to the AI analyzer.

That created two problems:

• obvious issues could be missed
• tokens were wasted on style observations instead of critical risks

The fix was architectural:

Static analysis runs first. AI reasons second.

This gave us:

• deterministic grounding
• lower hallucination risk
• more consistent results
• better cost efficiency

---

3) MuleSoft structure assumptions

The MuleSoft validator expects a real project layout (for example, src/main/mule/ and related project structure).

That becomes tricky when a user pastes only a single XML snippet.

We solved this by creating a temporary wrapper flow that:

• scaffolds the minimal expected structure
• runs the validator safely
• cleans up after execution

This let us support snippet-level review while still leveraging a real project validator.

---

4) Multi-agent latency

A sequential multi-agent system is naturally slower than a single prompt.

We improved responsiveness by:

• keeping prompts narrow and purpose-specific
• using async HTTP end-to-end
• limiting output shape aggressively
• making the optimizer a graceful fallback stage (if it times out, the user still gets a complete response)

---

Accomplishments that we’re proud of

1) Deterministic + AI hybrid architecture

PolyDev Coach does not rely on AI alone.

Critical issues such as hardcoded credentials are caught by deterministic static analysis before any AI token is spent.

That gives us:

• stronger trust
• lower hallucination risk
• better cost discipline

---

2) Self-validating multi-agent workflow

The independent validator is one of our proudest architectural choices.

Instead of assuming the refactor is correct, we:

• score it
• explain confidence
• optionally retry with feedback

That turns the system into a self-validating pipeline, which is much closer to what enterprise teams need.

---

3) Specialized MuleSoft support

Most AI coding demos focus only on mainstream languages.

PolyDev Coach stands out because it brings AI-assisted review to MuleSoft, where:

• reviewer scarcity is real
• integration bugs are costly
• domain-specific patterns matter

Combining open-source MuleSoft static analysis with Amazon Nova-powered coaching creates a much more practical enterprise tool.

---

4) Thoughtful Amazon Nova model routing

We intentionally used:

• Micro where speed and cost matter
• Lite where reasoning and coaching matter
• Pro where code generation quality matters

That made the architecture:

• technically stronger
• more cost-aware
• more explainable to judges

---

What we learned

1) Multi-agent systems only work well when each agent is narrow

The quality improved dramatically when each agent had exactly one job:

• analyze
• explain
• refactor
• validate
• optimize

Specialization improved:

• consistency
• prompt quality
• structured outputs
• reliability

---

2) Static analysis and AI are complementary

Static analysis gives:

• determinism
• trust
• reproducibility

AI gives:

• explanation
• prioritisation
• remediation
• coaching

The strongest architecture is not “AI replaces tooling.”

It is:

Deterministic tooling establishes ground truth. AI adds reasoning and remediation.

---

3) Trust matters more than raw generation

Developers do not just need code generated.
They need to trust:

• why an issue matters
• why a fix is safer
• whether the output is good enough

That is why the validator layer became central to the product.

---

What’s next for PolyDev Coach

We see PolyDev Coach evolving into a full enterprise developer workflow:

• VS Code extension — run reviews directly from the editor
• GitHub PR integration — post findings as PR review comments automatically
• Streaming UI — stream agent progress and partial results for better UX
• Repository-level analysis — scan broader project structures, not just snippets
• Historical quality dashboard — track code quality trends over time
• Expanded enterprise rule packs — deeper language and framework-specific review policies

Built With

• amazon
• amazon-web-services
• app
• bases
• bedrock
• boto3
• cloudfront
• fastapi
• knowledge
• mulesoft-package-validator
• nova
• opensearch
• python
• react
• runner
• s3
• serverless