Live Workflow Interviewer

Inspiration

Small businesses often want to introduce AI but do not know where it can safely help. The problem is not a lack of documents. It is that real work contains exceptions, judgment calls, and approval boundaries that employees rarely write down.

Traditional process interviews separate conversation from documentation: someone interviews the employee, reviews the recording later, draws a workflow, and asks the employee to correct it. The most valuable tacit knowledge often appears only when the employee sees a misunderstanding and says, “That is not quite right.”

Live Workflow Interviewer turns that correction loop into the product itself.

The workflow is not generated after the interview. The workflow is the interview.

What it does

A user describes any business process in English or Japanese while looking at a shared workflow canvas. The application:

• holds a natural, interruptible voice conversation;
• turns each completed user turn into a small, evidence-linked workflow update;
• distinguishes confirmed facts, AI inferences, unresolved decisions, and human approvals;
• lets the user correct the AI while the workflow is still being built;
• audits whether the current process is complete enough to design an AI-assisted version;
• produces a non-mutating after-view showing which steps remain human, which can be AI-assisted, where review is required, and what must be clarified first;
• generates an optional Codex-ready implementation brief and an owner-facing economic-impact estimate using only user-entered assumptions.

Supplier invoice processing is included as a reliable demo preset, but the product is not tied to that domain. Users can begin with a blank custom workflow.

How it works

1. Conversation as a shared interface

GPT-Realtime-2.1 manages low-latency speech-to-speech conversation over WebRTC. Voice is not used merely because it is faster than typing. It lets the user keep looking at the evolving workflow, interrupt naturally, point out errors, and remember exceptions without switching attention to a chat box.

2. Evidence-linked canonical workflow

GPT-5.6 Luna receives the latest accepted user turn and the current workflow state. It returns a strict Graph Patch rather than regenerating the whole graph. A deterministic reducer validates and applies operations such as adding a step, updating a decision, connecting a branch, or marking an unresolved condition.

Confirmed nodes must cite exact words from a real user turn. Assistant speech, invented utterance IDs, mismatched excerpts, stale revisions, and invalid topology are rejected.

3. Readiness and AI-assisted after-view

GPT-5.6 Sol audits a stable snapshot. If decisions, exceptions, evidence, or handoffs remain unclear, the user is sent back to the relevant part of the interview. Only a ready current revision can unlock the after-view.

Sol then creates a separate, evidence-linked AI-assisted design without changing the original workflow. This preserves the distinction between “how work happens today” and “how AI might help.”

How I built it

The project is a TypeScript and React web application deployed with OpenAI Sites. The browser owns the session-only canonical workflow. Realtime uses WebRTC, while server routes keep the standard OpenAI API key secret and handle Luna patches, Sol audits, the after-view, and the optional implementation package.

The application includes:

• structured model contracts and runtime validators;
• evidence provenance checks;
• stale-response and reset protection;
• fail-closed readiness gates;
• a collision-free scrollable workflow layout;
• English and Japanese interview instructions;
• text fallback for environments where microphone access is unavailable;
• offline HTML and Markdown result exports;
• per-runtime request throttling and documented project-budget limits.

No authentication or database was added. Session data remains in the current browser session unless the user explicitly downloads a report.

How Codex accelerated development

Codex was the main development partner throughout Build Week. It helped:

• turn the product concept into testable state and API contracts;
• implement the Realtime WebRTC connection;
• design the Graph Patch schema and deterministic reducer;
• build evidence validation and fail-closed behavior;
• identify race conditions around reset, overlapping model requests, and stale Sol results;
• create regression fixtures and automated tests;
• improve the workflow layout and bilingual interface;
• package and deploy the application to Sites;
• review the product from both user and judge perspectives.

Codex did not make the product decisions alone. I reviewed its proposals, rejected unauditable scoring and model-owned truth, and repeatedly narrowed the scope to one coherent experience.

Important human decisions

The key decisions I made were:

• use voice to maintain shared visual attention, not as a decorative input method;
• let Realtime facilitate conversation while Luna owns canonical workflow updates;
• require user evidence before an AI inference becomes a confirmed fact;
• update the graph through validated patches instead of replacing the whole workflow;
• keep Sol's AI-assisted design separate from the AS-IS workflow;
• fail closed when the workflow or transcript changes during an audit;
• keep data ephemeral and avoid authentication, persistence, and external business-system integrations for this focused demo;
• show economic impact as an assumption-based calculator rather than an unsupported AI claim.

Challenges

The hardest challenge was maintaining trust while several asynchronous systems were active. Voice, transcription, Luna patches, Sol audits, user corrections, and reset actions can all overlap. The application therefore uses revision checks, transcript identity checks, abortable requests, and generation tokens so that an old response cannot overwrite newer user intent.

Another challenge was translating messy conversation into a readable graph without inventing facts. Evidence-linked patches and a separate readiness model made the system slower than a single prompt, but much more auditable.

Accomplishments

I am most proud that the models have visibly different responsibilities:

• Realtime makes the conversation natural;
• Luna maintains the evidence-linked workflow;
• Sol challenges the workflow and designs the after-view;
• Codex helped build and test the product.

The result is not a chatbot, a transcript summarizer, or a one-click flowchart generator. It is a live process interview where human correction is part of the interface.

What I learned

Tacit knowledge does not necessarily require a perfect interviewer to ask every possible question. It often appears when a person sees an imperfect external representation and corrects it. Fast visual feedback changes the quality of the conversation.

I also learned that reliable AI products need explicit ownership of state. The model can propose a patch, but application code must validate evidence, topology, revision, and safety boundaries.

What's next

The next step is to test the interview with real small-business operators and consultants, then measure whether the live correction loop reduces follow-up interviews and exposes more decision boundaries than notes or transcripts alone. Future versions could add deliberate persistence, collaboration, and integrations, but only after validating this core experience.

Built With

• openai
• openai-api