CivOS

Test-fly a strategy before it meets the real world.

Comment

Inspiration

What it does

How we built it

Project Story

Inspiration

Important decisions rarely fail because people lack ideas. They fail because assumptions remain hidden, affected people are overlooked, evidence is mixed with intuition, and teams commit before they understand what could go wrong.

Most AI products respond to a decision with a polished recommendation. That can be useful, but it can also create false confidence. We wanted to build something different: an AI system that helps people test-fly a strategy before it meets the real world.

That idea became CivOS—a decision flight simulator for civic, organizational, product, and business tradeoffs. Instead of acting like an oracle, CivOS helps users clarify the decision, expose uncertainty, simulate alternative conditions, challenge assumptions, and identify when human approval is required.

What CivOS Does

CivOS turns an uncertain idea into an inspectable decision model.

The experience begins with a guided intake rather than an immediate AI answer. It anchors:

  • The real-world context
  • The strategy being tested
  • The intended outcome
  • Affected stakeholders
  • Constraints and non-negotiables
  • Available evidence
  • Relevant time horizons

If a critical gap remains, CivOS asks one high-value clarification question—the question most likely to change the recommendation.

Once the decision state is confirmed, CivOS runs a visible multi-perspective analysis:

  1. Operator develops an executable and reversible plan.
  2. Skeptic searches for failure modes and reversal conditions.
  3. Stakeholder identifies who benefits, who bears costs, and who lacks influence.
  4. Evidence Auditor separates supported claims from assumptions and missing evidence.
  5. Decision Manager synthesizes the independent outputs and explicitly references their contributions.

Rather than hiding this work behind a loading spinner, CivOS presents a Live Agent Run. Users can inspect each stage’s input, method, output, status, and duration.

The resulting decision workspace includes:

  • A live decision graph
  • Support, dependency, and conflict relationships
  • Evidence provenance
  • Counterfactual simulation
  • A one-click red-team challenge
  • Decision Memory and version history
  • Decision Diff
  • Uncertainty and readiness indicators
  • Guardrail receipts
  • Human approval gates
  • Downloadable action tools
  • Trace and evaluation metadata

CivOS avoids unexplained confidence percentages. Instead, it communicates uncertainty through interpretable measures such as evidence strength, unresolved unknowns, reversibility, material disagreement, and decision readiness.

How We Built It

CivOS is built as a lightweight web application with a deliberately inspectable architecture.

The frontend uses HTML, CSS, and JavaScript to manage the decision intake, interactive graph, Agent run visualization, evidence views, simulations, approval boundaries, and persistent local decision state.

The server layer exposes focused API routes for:

  • Decision intake structuring
  • Multi-perspective analysis
  • Evidence auditing
  • Uploaded-material extraction
  • Connection testing

The primary AI workflow uses the OpenAI Responses API with gpt-5.4-nano. The model receives decision-specific context and returns structured outputs for the specialist perspectives and Manager synthesis.

The orchestration pattern is:

User decision and materials
        ↓
Intake and clarification
        ↓
Decision Manager
        ↓
Independent specialist analyses
        ↓
Manager synthesis
        ↓
Structured decision state
        ├── Live graph and brief
        ├── Evidence and provenance
        ├── Challenge and simulation
        └── Human approval boundary

Evidence Mode uses model tools to distinguish between:

  • Verified — supported by an accessible source
  • User-provided — derived from supplied materials
  • Assumption — plausible but unverified
  • Contested — affected by meaningful disagreement
  • Missing — evidence still needs to be collected

Verified status is not granted from model memory alone. A claim must be connected to tool-returned evidence.

For privacy and deployment flexibility, CivOS supports two connection modes:

  • A managed server-side OpenAI connection
  • An optional user-provided OpenAI API key

A personal key remains only in the active browser page. It is not written to Local Storage, Decision Memory, Trace data, source code, or the public repository.

The application is deployed on Vercel, with the interface served as static assets and AI operations isolated in server-side functions. Production secrets are stored as encrypted environment variables.

Agentic Design

The most important design principle was to make agency visible and accountable.

CivOS does not present multiple chatbots side by side. Each specialist produces an independent structured result, and the Manager must synthesize those outputs without inventing specialist claims.

The system also demonstrates that responsible agents should know when to stop. Investigative and drafting tasks can proceed automatically, but external, costly, destructive, or scope-expanding actions require explicit human authorization.

This creates a clear boundary:

[ \text{Autonomous investigation} \neq \text{Autonomous execution} ]

CivOS can analyze, challenge, simulate, and prepare—but consequential actions remain under human control.

Challenges We Faced

Making the Agent process real

It is easy to create an animation that looks agentic. It is much harder to ensure that each visual event corresponds to a real model request, tool call, structured output, skipped capability, or authorization boundary.

We designed the interface to state honestly when a tool or model is unavailable instead of simulating successful work.

Avoiding false precision

Early decision interfaces often rely on attractive confidence scores. We found that a number such as “84% confident” is meaningless unless its basis can be inspected.

We replaced numerical confidence with evidence strength, unknowns, reversibility, readiness, and disagreement. This made the system feel less like a prediction engine and more like a serious decision instrument.

Balancing intelligence with usability

A detailed decision system can quickly become overwhelming. Our early flow asked users to complete too many fields manually and displayed advanced controls before users understood the product.

We redesigned the experience around progressive disclosure:

  • Start with an approachable landing page
  • Let users describe an idea in their own words
  • Automatically structure what can be inferred
  • Ask only one materially important question
  • Show advanced tools after the decision model exists

Building graceful failure modes

Model services, evidence tools, and external APIs can fail. CivOS needed to preserve the user’s work without pretending that an Agent had completed an analysis.

When AI is unavailable, the system keeps the structured decision state, marks missing capabilities explicitly, and offers retry or connection options.

Preserving user control

Approval gates initially risked feeling like obstructive pop-ups. We refined them so users can understand the requested action, edit its scope, reject it, or ask why authorization is required.

We also removed automatic transitions after an Agent run. When analysis finishes, CivOS waits for the user to click View decision details.

What We Learned

We learned that the quality of an AI decision product depends less on how confidently it answers and more on how clearly it exposes its reasoning boundaries.

A useful agent should be able to:

  • Recognize missing information
  • Ask a valuable question
  • Select appropriate perspectives
  • Use tools instead of relying on memory
  • Preserve disagreements
  • Explain what could change its conclusion
  • Distinguish preparation from execution
  • Stop for human authorization
  • Remember how the decision evolved

We also learned that provenance is not merely a compliance feature. It is a core interaction model. When users can inspect where a conclusion came from, which Agent challenged it, and what evidence could reverse it, the AI becomes easier to trust—and easier to disagree with productively.

What’s Next

The current version establishes the core decision workflow. The next stage is to turn the existing components into a unified, durable decision runtime.

Planned improvements include:

  • OpenAI Agents SDK orchestration and native tracing
  • A persistent structured Decision State
  • Account-based cross-device Decision Memory
  • Deeper File Search for policies, budgets, and meeting records
  • MCP integrations for organizational knowledge
  • Richer scenario recalculation
  • Calendar, Slack, and email approval actions
  • Collaborative decision review
  • Formal evaluation suites for evidence quality and stakeholder coverage
  • More explicit short-, medium-, and long-term causal modeling

Our long-term goal is not to automate important decisions. It is to help people make them with better evidence, clearer tradeoffs, visible uncertainty, and accountable human control.

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for CivOS

Built With

Share this project:

Updates