Inspiration
Marketing teams increasingly use AI agents to analyze performance and recommend budget changes. But generating a recommendation is only part of the production problem.
A campaign may have the highest historical ROAS and still lack sufficient evidence for immediate budget expansion. Historical correlation does not prove incrementality, and past efficiency does not guarantee that performance will remain stable at a larger scale.
I had already built a production-oriented marketing analytics agent with a Planner, a ReAct Executor, 13 analytical tools, grounded evidence, human-in-the-loop controls, and a 60-case evaluation framework. During OpenAI Build Week, I used Codex with GPT-5.6 to address the next bottleneck:
How can an organization determine what an AI recommendation is actually allowed to do?
That question inspired Campaign IC: an evidence-aware decision governance layer for the Smart Marketing Campaign Optimizer.
What it does
Smart Marketing Campaign Optimizer helps marketing teams move from raw campaign data to governed, auditable decisions.
The original analytical agent can answer questions about campaign performance, attribution, trends, audience value, data quality, and budget scenarios. Campaign IC begins after the analysis and evaluates four separate questions:
- What does the available evidence support?
- Which claims remain unsupported?
- What action does organizational policy permit?
- What still requires authorized human judgment?
In the reproducible demo, a marketer proposes increasing TikTok’s budget by 20% because it has the highest historical ROAS.
The system confirms the observation:
- TikTok has the highest observed ROAS at 12.1x.
- Current spend is $200,000.
- Observed revenue is $2.42 million.
- A static constant-performance scenario produces $2.904 million at $240,000 in spend.
However, Campaign IC does not treat this scenario as a forecast. It identifies two unsupported assumptions:
- Historical ROAS proves incremental impact.
- Performance will remain stable as spending increases.
Because the requested change exceeds both the percentage and dollar-risk thresholds, the system blocks a full rollout. It creates a limited action envelope instead:
Maximum Experimental Exposure = $40,000 × 15% = $6,000
The 15% value is explicitly treated as a policy-defined exposure cap—not an AI-generated optimal budget.
Because the policy does not define the experiment’s approved metric, duration, or pass/fail threshold, the final status is:
HUMAN_INPUT_REQUIRED
The system also generates an investment-committee-style memo and a machine-readable audit record containing the proposal, claims, evidence matrix, tool trace, SQL provenance, policy snapshot, triggered clauses, and final decision.
How we built it
The system is written in Python and builds on the existing Smart Marketing Campaign Optimizer architecture.
The original agent includes:
- A Planner for intent classification and task contracts
- A ReAct Executor
- 13 specialized marketing analytics tools
- SQLite-based campaign data
- Standardized tool outputs using
{status, insight, evidence, provenance} - Evidence-grounded response generation
- Human-in-the-loop escalation rules
- Session-level cost controls
- A 60-case evaluation framework
During Build Week, I used Codex with GPT-5.6 to inspect this architecture and design Campaign IC as a separate governance boundary after analytical reasoning.
The new layer includes:
- Proposal and claim normalization
- Evidence Matrix construction
- Evidence-sufficiency checks
- Unsupported-claim detection
- Versioned policy evaluation
- Materiality and risk-threshold gates
- Action-envelope generation
- Human-input requirements
- Structured IC memo generation
- Machine-readable audit records
For reliable judging, I also built a deterministic local adapter that follows the existing agent’s standardized tool-output contract. It uses a small reproducible SQLite fixture and read-only SQL queries, allowing the governance workflow to run without an API key.
Codex with GPT-5.6 helped me:
- Trace the existing system and identify the correct governance boundary
- Translate governance concepts into typed, testable modules
- Detect places where the system might overstate observational evidence
- Separate a policy risk cap from an optimized recommendation
- Design the deterministic judging path
- Implement auditability and provenance
- Write and validate the governance test suite
- Improve documentation and package the public repository
All 13 governance tests pass.
Challenges we ran into
The hardest challenge was not generating a budget recommendation. It was preventing the system from claiming more than the evidence could support.
A high historical ROAS is easy to calculate, but it does not establish causality. A static scenario is also easy to generate, but it should not be presented as a forecast. The system therefore had to distinguish carefully between:
- Observation and causal evidence
- Scenario calculation and prediction
- Risk limits and optimization
- Automated analysis and authorized judgment
A second challenge was keeping policy logic deterministic and auditable. If an AI system invents experiment success criteria or silently changes policy thresholds, the output may sound useful while becoming impossible to govern. Campaign IC therefore freezes the applicable policy snapshot, records every triggered clause, and refuses to invent missing approval criteria.
The third challenge was reproducibility. The full agent can use live model calls and a broader analytical tool graph, but judges need a fast and stable way to inspect the new governance layer. The local adapter had to preserve the real data contract and provenance structure without pretending to invoke components that the demo does not run.
Accomplishments that we're proud of
I am especially proud that Campaign IC does not simply add another recommendation layer. It creates a clear boundary between analysis and authority.
The system can now:
- Confirm supported observations without promoting them into causal claims
- Identify missing incrementality and scale evidence
- Block a materially risky full rollout
- Calculate the maximum policy-permitted experimental exposure
- Explain why the cap is not an optimized recommendation
- Stop when authorized human judgment is required
- Produce a complete machine-readable decision trail
- Run locally without credentials
- Pass all 13 governance tests
I am also proud that the system preserves the value of the original analytical agent while adding governance as a modular layer. This makes Campaign IC applicable beyond one marketing workflow.
What we learned
The most important lesson was that a production-ready agent needs more than reasoning and tools. It also needs an explicit model of authority.
Evidence, policy, and approval are different things:
- Evidence determines what the system can support.
- Policy determines what the organization permits.
- Human authority resolves decisions that the system is not authorized to make.
I also learned that “human in the loop” should not be a generic fallback. The system should state exactly what input is missing, why it matters, and who must authorize it.
Finally, working with Codex and GPT-5.6 showed me the value of using an AI coding agent not only to generate code, but also to challenge system boundaries, inspect claims, identify overreach, and convert abstract governance principles into reproducible software behavior.
What's next for Smart Marketing Campaign Optimizer
The next step is to connect Campaign IC directly to the live Planner, Executor, Aggregator, and full 13-tool analytical graph.
Future development will include:
- Organization-specific policy configuration and approval roles
- Integration with real campaign and experimentation platforms
- Stronger incrementality evidence from randomized or quasi-experimental designs
- Pre-registration of metrics, duration, and pass/fail thresholds
- Multi-stage approval workflows for different risk levels
- Continuous monitoring after a pilot is approved
- Automatic rollback when guardrails are violated
- Policy version comparison and decision replay
- Evaluation across more campaign types and governance scenarios
The long-term goal is to make AI recommendations operationally usable without making them operationally unaccountable.
Smart Marketing Campaign Optimizer is evolving from an agent that can answer “What should we do?” into a system that can also answer:
“What does the evidence justify, what does policy allow, and who is authorized to decide what happens next?”
Log in or sign up for Devpost to join the conversation.