Decora

• 

  Cover

Inspiration

A new kind of model API is showing up: native Decisions APIs. Chat completions decode tokens one at a time, and you hope the JSON parses. Native decision models such as OpenAI (/v1/decisions), Google Gemini (gemini-decision-1), TypeSafe (Jev) and OpenRouter (Alpha Decisions) work differently. They evaluate structured predicates, discrete choices and ordered rubrics in a single forward pass and return probabilities and confidence scores directly.

The trouble is that every provider shaped its API differently:

• OpenAI expects an ordered questions array of predicate, choice (option values may mix boolean and string) and score (levels indexed 0..N-1), plus role-based input messages (system, developer, user, assistant, input_json, input_image), authenticated with Authorization: Bearer.
• Google Gemini authenticates with x-goog-api-key. It takes a keyed questions map (noul, choice, score), limits choice option keys to 1–128 ASCII characters with string-only descriptions, requires string[] rubric criteria, and reports single-pass usage as total_input_tokens and input_tokens_by_modality.
• OpenRouter and TypeSafe (Jev) expect a top-level state string and keyed questions with noul and criteria. Their token accounting is different again (billable_input vs. provider_reported).

Some "unified gateways" quietly fall back to chat completions and make up the probabilities. We call these phantom probabilities. We built Decora to keep them out: a single-tenant Rust gateway that stitches all four native Decisions APIs together behind strict, verified contracts. It never uses chat-completion workarounds and never fabricates numbers.

What it does

Decora exposes four inbound endpoints, one per official dialect (plus GET /healthz):

Existing SDKs keep sending their native requests unchanged. Each endpoint strictly validates its own dialect: sending state to /v1/decisions returns 400 invalid_request. Any public model alias can then be routed to any compatible upstream.

1. Hub-and-spoke protocol translation (Dialect ↔ Canonical Decision IR ↔ Adapter)

   • Same-dialect passthrough. When the inbound dialect matches the upstream (e.g. Gemini → Gemini), Decora replaces only the top-level model. It keeps the original question keys, option keys and criteria, still verifies every number in the response, and returns the native response unchanged.
   • Cross-dialect translation and fallback. When a request crosses dialects, or a same-dialect primary fails with 429/5xx/timeout and falls back to a different provider, Decora switches per attempt to translation through the canonical IR. It maps predicate ↔ noul, ordered arrays ↔ collision-free keyed maps (q0..qN-1), and restores the original question order and nullable or duplicate names.
   • Multi-role message arrays and input_json payloads are encoded into a deterministic envelope (openai_decisions_text_v1) for upstreams that only accept flat text or state. Clients never flatten data or set conversion flags.
   • Typed choices survive the round trip: true vs. "true", false vs. "false", empty strings, and Unicode labels like "售后". Internally they travel as collision-proof keys (decora_choice_0..N-1).

2. Mathematical verification of every probability Decora never re-normalizes, clamps or invents upstream numbers. It checks every response before the client sees it:

   • Every probability $p_i$ and every confidence must be a finite number in $[0, 1]$.
   • For a choice question over options $C = {c_0, \dots, c_{k-1}}$, the distribution must cover $C$ exactly once, and $$\left| \sum_{i=0}^{k-1} p(c_i) - 1 \right| \le \epsilon, \quad \epsilon = 10^{-4}$$ The returned choice $c^$ must have the highest probability: $p(c^) = \max_i p(c_i)$.
   • For a score question with $N$ ordered levels $0, \dots, N-1$, Decora checks both the sum and the expected score $S$: $$\left| \sum_{i=0}^{N-1} p_i - 1 \right| \le 10^{-4} \quad \text{and} \quad \left| S - \sum_{i=0}^{N-1} i \cdot p_i \right| \le 10^{-4}$$ Any violation returns 502 upstream_protocol_error and is not retried, so bad data never reaches your business logic. The same checks apply in passthrough mode, including the score.legend and token-sum checks.

3. Honest usage accounting (compatible vs. strict) Decora doesn't fill missing cache or reasoning token counts with 0. In the default compatible policy, cross-dialect responses keep the upstream counts that can be verified, omit unknown breakdowns, and record where each number came from in usage.decora (billable_input for TypeSafe, total_input_tokens / input_tokens_by_modality for Gemini, provider_reported / provider_cost for OpenRouter). Same-dialect passthrough returns the native usage untouched. If a deployment needs exact usage schemas, an operator can set usage_policy: strict. Cross-dialect candidates that can't satisfy it are then rejected before any network call is made.

4. Capability-aware weighted routing with bounded fallback Candidates are filtered by real capabilities (question_types, images, prompt_cache, safety_identifier, max_questions, usage policy) and ordered by priority, then weighted sampling without replacement. Retries are bounded by a total max_attempts, and 429 Retry-After counts against the time budget.

How we built it (with Kiro)

Decora is written in Rust (2024 edition) with axum, tokio, reqwest (rustls, redirects and proxy auto-discovery disabled), serde, serde-saphyr, and subtle for constant-time key comparison.

Kiro shaped how the project was built:

• Steering as the single source of truth (.kiro/steering/). Five steering documents (product.md, protocol-contract.md, security.md, structure.md, tech.md) hold the architectural invariants. protocol-contract.md uses inclusion: fileMatch on {src/adapter.rs,src/dialect.rs,src/protocol.rs,src/gateway.rs,tests/offline.rs,docs/PROTOCOL.md}. Whenever protocol code is open in the Kiro IDE, Kiro automatically loads the dialect schemas, the passthrough rules, the probability tolerance ($\epsilon = 10^{-4}$) and the security boundaries. Those rules didn't have to be repeated in every prompt, and they didn't drift between sessions.
• Spec-driven development (.kiro/specs/). Each of the six milestones has a Kiro requirements.md → design.md → tasks.md spec: decisions-gateway, native-upstreams-and-usage, unified-contract-and-lowering, gemini-backend, multi-dialect-endpoints and dialect-passthrough. From gemini-backend onward we wrote the spec first, then had Kiro implement it task by task, checking off tasks.md as it went. (.kiro/ was accidentally git-ignored early in the project and re-tracked on Nov 13, which is why the folder's git history starts there.) The acceptance criteria are written as testable behaviour, such as "a tampered score.legend returns 502 and is not retried", and we turned them into assertions in the offline integration suite. Compared with vibe coding, the specs made large refactors safe, because the tests pinned the contract while the code underneath changed. The move from one endpoint to the four-dialect hub-and-spoke design was one such refactor.
• Agent hooks (.kiro/hooks/). rust-fmt-check-on-save.json runs cargo fmt --check every time a .rs file is saved. verify-reminder-on-stop.json is an agent hook that fires when Kiro finishes a turn. It requires the full pipeline (fmt --check, check --locked, test --locked, clippy -D warnings) to have actually run, and README, docs/PROTOCOL.md, config.example.yaml and HANDOFF.md to be in sync. It also forbids claiming a check passed if it wasn't run.
• The same .kiro/ folder works in Kiro CLI. Kiro CLI (V3) reads the same folder: /config lists the steering files (with protocol-contract as fileMatch) and both hooks. Asked about the probability tolerance, Kiro reads protocol-contract.md and answers 0.0001 with the file cited. The Stop hook fires in CLI sessions too: after answering, Kiro adds which checks it did not run and refuses to claim they passed.
• Handoff across agents. HANDOFF.md, together with each spec's tasks.md checklist, let Kiro sessions and subagents pick up complex protocol refactors from where the last one stopped without breaking the security or contract invariants.

Challenges we ran into

1. Type collisions and ASCII key limits across providers. OpenAI lets choice options mix booleans (true, false) with arbitrary strings ("true", "false", "", "售后"). OpenRouter and TypeSafe represent criteria as a JSON object, whose keys can only be strings, and Gemini limits option keys to 1–128 ASCII characters. Our fix was deterministic internal indexing (decora_choice_0..N-1) plus structured criteria serialization in the adapter. The exact typed value comes back on the way out without collisions, even when a client literally passes "decora_choice_0" as an option value.
2. Preventing false multimodal fallbacks. {"type": "input_json", "data": {"image_url": "https://..."}} is ordinary JSON data, not an image. When a request contains a real input_image part, a text-only adapter must never turn the URL into a string and pretend it looked at the image. We inspect the request structure before lowering it, so real images are filtered accurately while JSON fields with the same name pass through untouched.
3. Security and retry hazards. If a response body stream drops after a 200 OK header, retrying can bill the request twice. Ordinary HTTP client debug logs or error echoes can also leak API keys and user prompts. So we use constant-time inbound authentication (one credential per request, Bearer or x-goog-api-key), outbound clients that never follow redirects, a fixed error envelope ({"error":{"message","type","param":null,"code"}}), and a strict no-retry rule once upstream response headers have arrived.

Accomplishments that we're proud of

• Four dialects in, four native engines out. The same OpenAI-dialect request (system/user roles, nested input_json, mixed boolean and string choices) gets back the same answers contract whether it's served by the OpenAI, Gemini, OpenRouter or TypeSafe adapter: the same question order and names, the same typed choice values, and the same answer fields. Only the models' own probabilities differ, and every one of them is verified. Gemini, TypeSafe and OpenRouter clients can also call Decora in their own dialects, with passthrough and automatic cross-dialect fallback.
• A live end-to-end run. The demo script (demo/run.sh) sends the same OpenAI-dialect support-ticket request through Decora to real gpt-6-luna (OpenAI) and real gemini-decision-1 (Gemini): two different models, two different opinions, one answer shape, every number verified. The Gemini → Gemini passthrough also ran live with native keys preserved. TypeSafe and OpenRouter, the fallback, the phantom (Σp = 0.93 → 502) and the JSON-as-data cases run against labelled loopback mocks, and the video labels every row real API or loopback mock.
• An offline verification suite that costs nothing to run. 43 tests (42 end-to-end integration tests against loopback TCP servers that emulate each provider's official schema, plus 1 unit test) cover translation, passthrough, fallback, retry budgets, tampered probabilities and usage policies. The suite finishes in under a second. The full gate passes: cargo fmt --check, cargo check --locked, cargo test --locked, cargo clippy --all-targets -- -D warnings, and cargo audit across 182 locked crates with 0 vulnerabilities.
• Being explicit about limits. The docs separate what Decora guarantees (dialect validation, the answer contract, the probability checks) from where upstream semantics really differ (usage.decora provenance, role hierarchy vs. text encoding). They also state clearly that compatible usage is not a perfect copy of every native usage schema.

What we learned

• Native decision models are fast, cheap and inconsistent. Single-pass models like gemini-decision-1 and Jev produce no output tokens at all (output_tokens = 0), but their REST contracts differ in small, sharp ways: auth headers, rubric formats, token accounting.
• A canonical IR keeps the work linear. Adding the Gemini/TypeSafe/OpenRouter inbound dialects meant writing one dialect module each, not $M \times N$ pairwise converters. Passthrough fell out naturally as the case where the dialect and the adapter are the same.
• Spec-first collaboration between agents works. With the rules in .kiro/steering/ and progress tracked in .kiro/specs/<feature>/tasks.md and HANDOFF.md, Kiro and its subagents could work together through large protocol refactors without drifting from the security or contract invariants.

What's next for Decora

• Live coverage for every provider. OpenAI and Gemini have been exercised live through Decora (see below); TypeSafe and OpenRouter are so far verified only against schema-accurate mocks. Next: live runs on those two, plus a billing cross-check on all four.
• Native image mapping for Gemini. Verified data:image/...;base64 payloads should map directly to Gemini inline_data when crossing dialects, with explicit MIME checks.
• Privacy-preserving OpenTelemetry metrics. Export latency, fallback counts and error-code counters without ever logging payloads or prompts.
• Production hardening. A TLS reverse proxy, rate limiting and process supervision guide for public deployments.

Built With

• api-gateway
• axum
• gemini
• kiro
• openai
• openrouter
• reqwest
• rest-api
• rust
• rustls
• serde
• tokio
• typesafe
• yaml