ObserveOS — The Self-Improving Clinic Operating System

I am a practicing healthcare professional running a solo clinic. Since February 2026, I have been systematically turning recurring problems from real clinical casework and daily operations into explicit, testable workflows that AI can assist without being allowed to invent or silently rewrite the record.

ObserveOS grew from that work, case by case and correction by correction. It was not invented as a hackathon concept. It is the result of pushing GPT-5.4, GPT-5.5, and the GPT-5.6 family against the limits of real longitudinal work, then converting failures, expert corrections, and useful behaviors into evidence boundaries, review gates, and regression tests.

For OpenAI Build Week, I extracted the CaseAgent Reflection Loop into a synthetic-only, independently runnable public implementation. Judges can inspect the core mechanism without any patient data: one four-round interactive case and two focused fictional conformance cases show how reports, observations, inferences, unknowns, practitioner answers, and stale analyses remain traceable.

Inspiration

Most AI products optimize for the next answer. Longitudinal professional work has a harder problem: information arrives over time from sources with different authority.

A client report is not a practitioner observation. A transcript is not automatically case truth. An inference is not evidence. An AI question may help recall, but the question itself must not seed the next conclusion. “I do not know” may be the most accurate result.

In the synthetic demo, the client reports that a step-down feels less stable near the end of a short set. A fluent note could silently upgrade that into an observation that control deteriorated. ObserveOS retains what was actually provided: this is still a client report. The AI may ask whether direct observation occurred, but that question remains outside evidence. Only the practitioner’s answer enters evidence, and the previous analysis becomes stale before it can be treated as current.

The work expanded beyond case review into intake, governed transcription, source-separated knowledge, operations readback, websites, campaigns, and content workflows. The recurring lesson was that a stronger answer was not enough; the workflow needed an explicit contract for what an answer was allowed to claim.

What we built during OpenAI Build Week

ObserveOS existed before Build Week as a private, evolving whole-practice workflow. During Build Week, Codex and GPT-5.6 were used to build a new, synthetic-only and independently runnable public implementation of its evidence-governance core:

• a local browser application and standard-library Python service;
• an append-only case event ledger with a tamper-evident hash chain;
• an evidence-only projection that excludes AI questions;
• practitioner-answer provenance, stale-analysis invalidation, and formal-save gates;
• deterministic Replay with no model, account, API key, or package-install dependency;
• optional live GPT-5.6 review through an existing Codex ChatGPT sign-in;
• a visible five-check Judge Tour whose progress is derived from the actual append-only event ledger;
• 47 automated contract tests, a four-round gold replay, a three-case synthetic governance corpus, a privacy audit, and JavaScript syntax verification.

The public CaseAgent Reflection Loop is the runnable Build Week project. The private operational systems are its lineage and product context, not hidden dependencies required by the judge.

What it does

The browser edition follows one fictional case through four rounds: a client report, a practitioner observation, an intervention with immediate retest, and a next-day follow-up.

Judges can also run two focused, fully fictional conformance cases. One tests whether a client report and practitioner observation remain separate when they appear to conflict. The other tests whether later non-replication makes an earlier analysis stale and weakens the earlier bounded inference. These public cases encode governance patterns only; they are not de-identified exports of private cases.

For each round, ObserveOS:

1. appends the new source without overwriting earlier history;
2. keeps supported findings, bounded inference, and explicit unknowns separate;
3. permits at most one bounded reflection question that cites existing evidence events;
4. keeps the AI question in interaction history and outside the evidence projection;
5. records the practitioner’s answer as new evidence with provenance;
6. marks the previous analysis stale when new evidence arrives;
7. reruns from the expanded evidence set; and
8. allows formal saving only when the current analysis and required reflection gates are satisfied.

RAG retrieves related text. An AI scribe summarizes a session. ObserveOS governs whether information may enter evidence, whether an analysis is still current, and whether the result is eligible for formal saving across time.

The broader ObserveOS product coordinates real modules for intake, governed transcription, CaseAgent work, source-separated knowledge, operations readback, websites and campaigns, and Social／VoiceLab workflows. Those modules keep separate truth owners rather than merging case evidence, knowledge, operations, and public content into one uncontrolled database.

The broader private CaseAgent workflow also maintains a versioned, practitioner-authored clinical decision logic base (v3.5.1 at submission time). It encodes support, weakening, and falsification conditions; retest and safety gates; and limits on what each input quality can support. The public Build Week kernel does not publish those proprietary clinical rules. It demonstrates their governing boundary: expert knowledge may guide questions and interpretation, but it does not become case evidence by itself.

“Self-improving” is human-governed, not autonomous self-modification. Practitioner answers and later evidence change the next governed run immediately. Proposed rules, gold cases, and regression tests enter the formal contract only after human review.

Try it in 90 seconds

Run .\START_DEMO.ps1 on Windows or python app.py --open-browser on any supported platform with Python 3.11+ and a modern browser. No package install is required.

Choose 90-second judge tour and keep Replay selected. Run the first evidence review, add the demo practitioner answer as evidence, and watch the tour verify the report boundary, AI-question exclusion, and stale-analysis gate from actual ledger state. Continue through the remaining source rounds and save the current analysis when all five checks are verified.

For broader deterministic proof, run python scripts/run_governance_corpus.py. It verifies three fictional cases, eight source rounds, fifteen analyses, seven practitioner answers, three current-analysis snapshots saved without regeneration, and three independent hash chains. The command requires no model account, API key, package install, or private data.

Replay is the primary judge route and requires no model access. Optional live mode reuses an existing ChatGPT-authenticated Codex session; availability may vary by account entitlement.

How we built it

The app is a local, dependency-free Python service with a responsive HTML, CSS, and JavaScript interface. Its domain layer uses append-only JSONL events with event IDs, sequence numbers, idempotency keys, previous hashes, and current hashes.

The evidence projection includes governed source events and practitioner answers while excluding AI questions. Supported findings and inferences must cite existing evidence event IDs; semantic interpretation remains human-reviewed. New evidence invalidates the prior analysis. The save gate stores the current normalized analysis without a second model call.

Live analysis launches codex exec through an existing ChatGPT-authenticated Codex session. The child process is ephemeral, read-only, schema-bound, isolated from repository rules, and receives only the fictional evidence projection through a sanitized environment. Unknown evidence IDs fail validation instead of being displayed as grounded output.

Replay implements the same demonstrated evidence-state transitions without requiring model entitlement. The public package includes a JSON output schema, deterministic fixture, gold evaluation, contract tests, and privacy release checks.

How we used Codex and GPT-5.6

Codex and GPT-5.6 were used to inspect the system lineage, separate formal truth from supporting material, design the evidence taxonomy, define reflection and save gates, implement the multi-round event model and browser application, and turn recurring failure modes into tests.

Across the broader development history, Terra was evaluated through isolated shadow replays, Luna became the locked daily runtime, and Sol handled the highest-value architecture and adjudication work. During Build Week, the central contribution was architectural: moving from “generate a better answer” to “build a mechanism that governs what an answer is allowed to claim.”

Primary Codex development session ID: 019f6743-1883-7b93-9f61-4a34a96d6e1f

No OpenAI API key was created for the application. Live mode reuses Codex ChatGPT sign-in; Replay makes no model call.

Seven-model internal benchmark

The latest fully comparable internal run evaluated seven model routes on the same seven strict, text-only Gold cases: 49 independent turn-by-turn chats and 462 canonical model calls. The two newest cases in the current nine-case private corpus were added later and are not included in these scores.

Model route	Average /100	Range	Sanitized practitioner assessment
Sol Medium	91.43	87–95	Most stable evidence boundaries and long-context revision; still requires human review.
Terra Medium	88.57	85–92	Strong and consistent; effectively tied with Luna Medium at this benchmark's resolution.
Luna Medium	88.43	78–93	Strong, concise, and source-faithful, with fewer proactive questions in some cases.
Luna XHigh	86.14	75–93	Strong on shorter cases, but less stable than Medium in the longest case.
Kimi K3	68.29	58–78	Found the main thread but needed intensive supervision for invented schedules and mechanism overreach.
GLM-5.2 Max	62.00	49–70	Frequently promoted working hypotheses into causal explanations.
DeepSeek V4 Pro Thinking	57.14	42–65	Highest correction burden and strongest long-context error recurrence.

These are internal practitioner-authored workflow-rubric scores, not clinical efficacy scores or a claim about every model capability. The full sanitized benchmark table includes per-case scores, correction burden, methodology, and limits.

Challenges we ran into

The hardest challenge was preventing fluent language from outrunning evidence.

We had to make several boundaries executable and visible:

• a report versus an observation;
• an observation versus a mechanism;
• an AI question versus human evidence;
• a cited event ID versus human-reviewed semantic support;
• an unanswered gap versus an explicit unknown;
• a current normalized analysis versus a second hidden generation at save time; and
• a whole-practice operating system versus one uncontrolled truth store.

We also needed a public implementation that demonstrated the governing idea without exposing private cases, recordings, operational records, credentials, or proprietary clinical rules.

Accomplishments that we are proud of

• A realistic multi-round same-case flow instead of a single prompt-response demo.
• AI questions are excluded from the evidence projection.
• Practitioner answers retain provenance and invalidate stale analyses.
• Unknown is preserved as a first-class state.
• Supported findings and inferences must cite existing evidence event IDs.
• Formal saving stores the current normalized analysis without a second model call.
• The append-only ledger is protected by a verifiable, tamper-evident hash chain.
• Replay reproduces the core workflow without account entitlement.
• Live mode uses an existing Codex ChatGPT sign-in with no API-key setup.
• A visible Judge Tour verifies five governance transitions from actual ledger state rather than a static checklist.
• Two additional public fictional cases test source-role conflict and later-evidence inference revision.
• Forty-seven automated tests, a four-round gold replay, a three-case governance corpus, a privacy audit, and JavaScript syntax verification.

Known limits

• Citation validation confirms that referenced evidence event IDs exist; semantic support remains human-reviewed.
• A reflection question must provide a non-empty case anchor and valid evidence IDs, but the prototype does not yet compare the anchor text semantically with the source.
• Custom browser text is user-declared synthetic and is not automatically de-identified.
• The local hash chain is tamper-evident, not externally notarized.
• The current save path stores the normalized analysis without a second model call; it is not yet bound to a reviewer identity or UI-render digest.
• This is a local, single-process prototype and is not autonomous diagnosis or medical advice.

What we learned

The most valuable AI behavior is not always answering. In high-context work, reliability comes from preserving where information came from, exposing uncertainty, asking one bounded question, and waiting for the human to decide what may enter the reviewed record.

We also learned that a whole-practice operating system does not need one monolithic truth store. It needs explicit handoffs, separate source authority, and a visible boundary between evidence, inference, and unknown.

What’s next

Next we will add explicit correction and supersession events, source-role compatibility checks, stronger source-anchor validation, reviewer-bound snapshot verification, a broader synthetic corpus, and measured comparisons against prompt-only and generic summarization baselines.

We will also connect the public Reflection Loop to broader ObserveOS modules through the same source-separated contracts. The fully integrated real-time in-clinic copilot remains the next product stage, not a capability claimed by this submission.

We welcome collaborators working on synthetic or properly de-identified small-practice pilots, workflow integrations, and AI evaluation of provenance and longitudinal human review. Human review will remain required before formal saving.

Built With

• codex
• event-sourcing
• gpt-5.6
• human-in-the-loop
• python