Cosu

Turn what you discover in AI conversations—and what you struggle to express—into precise English you can truly own.

Comment

Inspiration

Over eight years of teaching English, I repeatedly saw the same problem: many learners had meaningful ideas, but they could only express them with broad words such as good, bad, happy, difficult, or interesting.

The problem was not simply that they lacked advanced vocabulary. Often, they had not yet identified the exact meaning they wanted to communicate.

When a learner described an experience as “bad,” they might actually mean that it was disappointing, humiliating, exhausting, isolating, or unfinished. Simply replacing bad with a more sophisticated synonym did not solve the deeper problem.

I encountered a surprisingly similar experience during eighteen months of personal psychoanalytic work.

At one point, I learned the concept of mirroring. Before I had language for it, I could sense that something was missing in certain interactions, but I could not clearly identify or describe it. Learning the word alone was not enough either. It only became meaningful when I could connect it with an experience I had actually lived through.

That experience made me realize:

Precise language does not only help us communicate clearly. It can change what we are able to notice, organize, and understand.

When a vague experience receives a more accurate name, it can become easier to examine instead of simply being overwhelmed by it. I began to see a connection between emotional clarification and second-language learning: in both cases, progress often begins by moving from a broad label toward a more precise meaning.

At the same time, people are now having increasingly thoughtful conversations with AI. These conversations contain useful expressions, clearer formulations, and moments when users are trying—but struggling—to articulate something important.

Yet most of that language disappears when the conversation ends.

I built Cosu to connect these ideas:

Turn what you discover in AI conversations—and what you struggle to express—into precise English you can own.

Cosu is not a therapy product. It is an English-speaking and expression-training agent. Personal feelings provide one powerful use case, but the same process can help users express academic opinions, professional experiences, personal stories, and everyday ideas.

What it does

Cosu helps learners move through three stages:

Capture

During an AI conversation, Cosu identifies two kinds of learning opportunities:

  • useful expressions in the AI’s responses;
  • vague, broad, or incomplete ideas in the learner’s own speech.

Instead of saving isolated vocabulary, Cosu preserves the original context, the idea the learner was trying to express, and why the expression matters to that person.

Clarify

When a learner says something broad, such as:

“I felt bad about coming back.”

Cosu does not immediately rewrite the sentence or provide a list of advanced synonyms.

GPT-5.6 first identifies the unresolved dimensions of meaning. In this example, bad might refer to regret, shame, grief, disappointment, uncertainty, or the feeling of being trapped.

Cosu then asks one or two questions designed to distinguish between these possible meanings.

For example:

“Do you regret the decision itself, or are you grieving the life you might have had if you had stayed?”

Based on the learner’s response, Cosu creates an Expression Ladder:

Initial expression

I felt bad about coming back.

More specific

I regretted coming back.

Meaningfully precise

I was not simply regretting the decision. I was grieving a version of my life that ended before I had given it a real chance.

The final sentence is not selected merely because it sounds more advanced. It is selected because the learner confirms that it reflects the intended meaning more accurately.

Own

Understanding a better sentence is not the same as being able to use it independently.

Cosu therefore creates a new speaking situation and asks the learner to express the same underlying idea without seeing or copying the model sentence.

GPT-5.6 evaluates whether the learner has successfully transferred the meaning into the new context—not merely whether a target phrase appears in the transcript.

Cosu’s acoustic scoring system separately evaluates aspects of spoken delivery such as fluency, pauses, pronunciation, and intelligibility.

The result is then stored in the learner’s Personal Expression Memory, including:

  • the original conversational context;
  • the learner’s initial wording;
  • the clarified meaning;
  • useful language discovered in the AI conversation;
  • the final precise expression;
  • the learner’s own recording and example;
  • whether the expression has been captured, clarified, practised, or independently reused.

Cosu is not designed to create a larger vocabulary list.

It is designed to build a personal language system based on what the learner has genuinely tried to understand and communicate.

How we built it

Cosu existed before Build Week as an AI-powered spoken-English practice product. Its existing features included voice-based answers, transcription, language feedback, and an independently developed acoustic scoring API.

For Build Week, I focused on building a new Capture–Clarify–Own workflow around GPT-5.6.

The system combines three layers.

GPT-5.6 semantic reasoning

GPT-5.6 is used to:

  • understand the wider conversational context;
  • identify vague or underspecified meaning;
  • distinguish between several plausible interpretations;
  • generate a small number of useful clarification questions;
  • build an Expression Ladder;
  • extract personally relevant expressions from AI responses;
  • evaluate whether meaning has been transferred into a new speaking context.

This is not a synonym-replacement task. The model must reason about what the learner may be trying to express while still allowing the learner—not the AI—to confirm the final meaning.

Cosu acoustic scoring API

Cosu’s existing acoustic API evaluates the spoken layer independently from GPT-5.6.

The semantic layer evaluates:

Did the learner express the intended meaning?

The acoustic layer evaluates:

Could the learner deliver that meaning clearly and fluently?

Keeping these layers separate helps Cosu distinguish between knowing what to say and being able to say it aloud.

Personal Expression Memory

I designed a structured memory schema that connects each expression to:

  • its source conversation;
  • the learner’s original intention;
  • semantic clarification;
  • the learner’s own speech;
  • transfer performance;
  • future review and reuse.

This allows Cosu to track more than vocabulary difficulty. Over time, it can identify concepts a learner repeatedly describes vaguely, expressions the learner understands but cannot actively produce, and meanings that have already been transferred across multiple speaking situations.

Codex

I used Codex to help build and iterate on:

  • the new conversational workflow;
  • structured GPT-5.6 outputs;
  • the Expression Ladder schema;
  • Personal Expression Memory;
  • semantic-transfer evaluation;
  • integration between language reasoning and acoustic scoring;
  • testing and debugging of edge cases.

Build Week Evidence

Project Baseline

  • Evidence log created: 2026-07-20
  • Prototype working directory: /Users/heliang/COSU1
  • Original Gemini prototype baseline commit SHA: UNKNOWN - exported folder had no prior recoverable Git history
  • Verification note: on 2026-07-20, COSU1 was discovered as an untracked directory inside the parent Git repository rooted at /Users/heliang. The parent repository HEAD at discovery time was a69b6e65602c28ee41562af1915e5166d2db4b83, but that commit does not version the COSU1 directory itself. The original prototype SHA must be filled in later if the source repository or export metadata is recovered.
  • Git provenance verification executed on 2026-07-20:
    • git rev-parse --show-toplevel returned /Users/heliang.
    • git ls-files --stage -- COSU1 returned no tracked files for this project directory.
    • Therefore, no verifiable baseline commit SHA exists yet for the exported prototype folder itself.
  • Repository state after Milestone 1 confirmation:
    • /Users/heliang/COSU1 is now an independent Git repository.
    • Milestone 1 baseline commit SHA: 9383a190697cc3cf1e5895c566adecf0f03111e7
    • Milestone 1 tag: build-week-m1-baseline
  • Prototype provenance verifiable in code:
    • README contained AI Studio boilerplate and GEMINI_API_KEY setup instructions before Milestone 1 cleanup.
    • UI structure reflected a static visual prototype with local demo transcripts and browser speech-recognition fallback.

Milestone Log

2026-07-20 - Milestone 1: Clean up and engineer the Gemini prototype

  • Goal:
    • Remove misleading boilerplate.
    • Add Build Week evidence and architecture docs.
    • Create a minimal engineering baseline with lint, type-check, test, and production build commands.
    • Preserve the current visual design while making prototype limits explicit.
  • Files added:
    • .gitignore
    • docs/build-week-evidence.md
    • docs/architecture.md
    • docs/evaluation-plan.md
    • docs/demo-script.md
    • scripts/lint-build-week.mjs
    • src/components/BuildWeekStatus.tsx
    • src/lib/prototype-data.ts
    • tests/prototype-data.test.ts
  • Files updated:
    • README.md
    • package-lock.json
    • package.json
    • src/App.tsx
    • src/components/ChooseQuestion.tsx
    • src/components/Clarification.tsx
    • src/components/FirstResponse.tsx
    • src/components/ImportConversation.tsx
    • src/components/LandingPage.tsx
    • src/components/SecondResponse.tsx
    • vite.config.ts
  • Core feature commit SHA mapping:
    • 9383a190697cc3cf1e5895c566adecf0f03111e7 - Milestone 1 engineering baseline
  • OpenAI API / model usage actually present in this milestone:
    • None yet.
    • No OpenAI API call, model name, prompt, or response handling code has been added in Milestone 1.
  • Technical decisions:
    • Centralize demo copy and prototype fixtures in src/lib/prototype-data.ts.
    • Add a lightweight repository-specific lint script instead of introducing a new lint dependency during cleanup.
    • Add a small tsx --test suite so test execution is real and reproducible without a larger test framework yet.
    • Add an in-app status panel to distinguish implemented features from planned work.
  • Commands run:
    • git rev-parse --show-toplevel
    • git ls-files --stage -- COSU1
    • npm uninstall @google/genai
    • npm run lint
    • npm run typecheck
    • npm run test
    • npm run build
    • npm run lint
    • npm run typecheck
    • npm run test
    • npm run build
  • Real results:
    • git rev-parse --show-toplevel returned /Users/heliang.
    • git ls-files --stage -- COSU1 returned no tracked files for this project folder.
    • npm uninstall @google/genai succeeded: removed 36 packages and updated package-lock.json.
    • First npm run lint passed.
    • First npm run typecheck passed.
    • First npm run test failed before assertions due sandbox/runtime behavior in tsx --test:
    • Error: listen EPERM ... /var/folders/.../tsx-501/...pipe
    • First npm run build passed.
    • Fix applied: changed package.json test script from tsx --test tests/**/*.test.ts to node --import tsx --test tests/**/*.test.ts.
    • Second npm run lint passed.
    • Second npm run typecheck passed.
    • Second npm run test passed:
    • # pass 3
    • # fail 0
    • # duration_ms 330.587417
    • Second npm run build passed:
    • dist/index.html 0.41 kB │ gzip: 0.28 kB
    • dist/assets/index-sT8lYbFB.css 42.52 kB │ gzip: 7.51 kB
    • dist/assets/index-CYtyGu88.js 417.79 kB │ gzip: 125.27 kB
    • ✓ built in 2.85s
  • Known limitations after Milestone 1:
    • Conversation analysis is still driven by local prototype fixtures rather than OpenAI structured output.
    • No persistent storage, user analytics export, or evaluation dataset exists yet.
    • Acoustic scoring is not implemented.
    • Prototype-era UI copy and demo flow are preserved intentionally; product logic is still mostly simulated.
  • Codex session / feedback IDs:
    • Codex session ID: PENDING
    • Feedback ID: PENDING
  • Provenance split:
    • From original prototype:
    • Overall visual design, multi-step flow, demo transcript concept, browser speech-recognition fallback, expression-tree visualization.
    • Implemented by Codex in Milestone 1:
    • Build Week documentation set.
    • Engineering check scripts.
    • Centralized prototype data module.
    • Minimal automated tests.
    • UI disclosure of implemented vs not-yet-implemented capabilities.

Commit Mapping

  • Milestone 1 engineering baseline: 9383a190697cc3cf1e5895c566adecf0f03111e7
  • Milestone 2 structured analysis implementation: 719de5f
  • Milestone 3 first spoken response + expression tree: d79feab4ada768d79620c536e4255347122e3ffb
  • Latest committed pre-audit baseline on this branch: a3b14b20442846b3d225a23d9d0f1fe42d26f3a1

Test Evidence

  • Latest lint result: PASS on 2026-07-20 via npm run lint
  • Latest type-check result: PASS on 2026-07-20 via npm run typecheck
  • Latest test result: PASS on 2026-07-20 via npm run test with 28 passing tests
  • Latest production build result: PASS on 2026-07-20 via npm run build

2026-07-21 - Final submission audit before code freeze

  • Audit goal:
    • Run the full product path through Done.
    • verify lint, type-check, tests, and production build
    • identify remaining demo dependencies, production-safety issues, and documentation drift
    • update repository docs to match the real dirty worktree
  • Actual runtime path exercised:
    • Start -> Import -> Choose -> Speak -> Clarify -> Own It -> Done
    • exercised through the in-app browser using demo mode
    • End Prototype Session now reaches a completion page
  • Commands run:
    • npm run lint
    • npm run typecheck
    • npm run test
    • npm run build
    • npm run dev
  • Real results:
    • npm run lint passed
    • npm run typecheck passed
    • npm run test failed with 36 passing tests and 1 failing test
    • failing test:
    • clarification propose-inner-state copy keeps unsure path unconfirmed
    • failure cause:
    • the test still expects the older clarify copy path containing I'm not sure yet, but the runtime now moves directly into the expression-selection stage after the confirm step
    • npm run build passed
    • dist/index.html 0.41 kB │ gzip: 0.28 kB
    • dist/assets/index-DifdneuP.css 41.99 kB │ gzip: 7.47 kB
    • dist/assets/index-RhVDQQYA.js 518.56 kB │ gzip: 152.61 kB
    • Vite emitted a large-chunk warning above 500 kB, but the build completed successfully
  • Dirty worktree status at audit time:
    • modified tracked files:
    • server/analyze-first-response.ts
    • server/app.ts
    • server/prompts/first-response-tree-v1.ts
    • server/transcribe-first-response.ts
    • src/App.tsx
    • src/components/Clarification.tsx
    • src/components/ExpressionTree.tsx
    • src/components/SecondResponse.tsx
    • src/components/SpeakPage.tsx
    • src/lib/prototype-data.ts
    • src/session-state.ts
    • src/types.ts
    • tests/first-response-flow.test.ts
    • untracked feature files:
    • server/clarify-meaning.ts
    • server/generate-target-expressions.ts
    • server/prompts/clarify-meaning-v1.ts
    • server/prompts/target-expressions-v1.ts
    • tests/clarification-flow.test.ts
  • Demo-data audit:
    • only-for-demo fixtures:
    • DEMO_IMPORT_TRANSCRIPT
    • DEMO_ANALYSIS_RESULT
    • DEMO_FIRST_RESPONSE
    • DEMO_FIRST_RESPONSE_DURATION_MS
    • DEMO_FIRST_RESPONSE_TREE_ANALYSIS
    • DEMO_CLARIFY_SELECTED_BRANCH
    • DEMO_CLARIFY_MEANING_RESULT
    • DEMO_TARGET_EXPRESSIONS_RESULT
    • createDemoAnalysisResponse()
    • createDemoFirstResponseAnalysisResponse()
    • createDemoClarifyMeaningResponse()
    • createDemoTargetExpressionsResponse()
    • runtime still incorrectly relying on hardcoded or prototype-era logic:
    • src/components/SecondResponse.tsx
      • first-attempt comparison text is still hardcoded instead of using the actual first transcript from session state
      • vocal-clarity result is still hardcoded as Vocal Integrity: Strong
      • transformation comparison remains simulated, not model-evaluated
      • saved expression writes only to browser localStorage
      • the current simulated second response uses a generic sentence template and can produce awkward phrasing such as I often self-conscious...
    • src/components/ImportConversation.tsx, src/components/SpeakPage.tsx, and src/components/LandingPage.tsx
      • demo controllers remain visible in the main runtime
  • Production-visibility audit against requested hidden surfaces:
    • Design System Notes
    • fail
    • the header button is always rendered in src/App.tsx
    • Build Week Status
    • pass by default
    • rendered only when VITE_SHOW_BUILD_WEEK_STATUS === 'true'
    • prototype controllers
    • fail
    • demo entry and simulation controls are still exposed in the normal runtime
    • tree inspectors
    • partial / fail
    • the expression tree itself is a product surface, but there is still an interactive-map / clickable-inspector style behavior rather than a production-locked presentation
    • debug JSON
    • pass by default
    • the structured payload inspector exists only inside BuildWeekStatus
  • Secrets / privacy / log-risk audit:
    • .env.example is tracked and safe
    • local live credentials should be injected through the shell or deployment secret manager rather than stored in a shared workspace file
    • server-only environment variables referenced in code:
    • OPENAI_API_KEY
    • OPENAI_MODEL
    • OPENAI_BASE_URL
    • OPENAI_PROXY_URL
    • HTTPS_PROXY
    • HTTP_PROXY
    • OPENAI_TIMEOUT_MS
    • OPENAI_MAX_RETRIES
    • OPENAI_MAX_OUTPUT_TOKENS
    • OPENAI_FIRST_RESPONSE_MAX_OUTPUT_TOKENS
    • OPENAI_CLARIFY_MAX_OUTPUT_TOKENS
    • OPENAI_TARGET_EXPRESSIONS_MAX_OUTPUT_TOKENS
    • OPENAI_REASONING_EFFORT
    • OPENAI_TRANSCRIPTION_MODEL
    • DEBUG_LOG_PRIVATE_CONVERSATIONS
    • PORT
    • NODE_ENV
    • logging risk summary:
    • default logging is metadata-only
    • full imported conversation text is loggable only when DEBUG_LOG_PRIVATE_CONVERSATIONS=true outside production
    • raw transcript text is not logged by default
    • raw audio bytes are not logged by default
    • request IDs and response IDs are logged when available
    • server/index.ts logs the local server URL and NODE_ENV
  • OpenAI usage actually present in the audited build:
    • API:
    • OpenAI Responses API
    • OpenAI Audio Transcriptions API
    • models:
    • gpt-5.6
    • gpt-4o-mini-transcribe
    • prompt versions:
    • conversation-analysis-v1
    • first-response-tree-v1
    • clarify-meaning-v1
    • target-expressions-v1
  • Shared schema surface actually present:
    • conversation analysis:
    • conversationSummary
    • explorationHypotheses
    • discussionQuestions
    • first response tree:
    • eventNodes
    • meaningNodes
    • emotionHypotheses
    • broadExpressions
    • clarify meaning:
    • distinctionQuestion
    • meaningOptions
    • proposedInnerState
    • innerStateAlternatives
    • targetMeaningDraft
    • target expressions:
    • candidates[].id
    • candidates[].phrase
    • candidates[].category
    • candidates[].rationale
    • candidates[].example

OpenAI Usage Ledger

  • Live OpenAI API usage was executed on 2026-07-20 through:
    • scripts/diagnose-openai.ts
    • server/analyze-conversation.ts
    • server/transcribe-first-response.ts
    • server/analyze-first-response.ts
  • API:
    • OpenAI Responses API
    • OpenAI Audio Transcriptions API
  • Model:
    • gpt-5.6
    • gpt-4o-mini-transcribe
  • Prompt / request types actually exercised:
    • Minimal GET /v1/models connectivity check
    • Minimal POST /v1/responses text request
    • OpenAI SDK minimal responses.create(...) request
    • OpenAI SDK minimal structured-output request
    • Full Milestone 2 conversation-analysis request with conversation-analysis-v1
    • Full Milestone 3 first-response tree request with first-response-tree-v1
    • Milestone 3 transcription request with uploaded audio bytes
  • Server-only configuration verified in code:
    • OPENAI_API_KEY
    • OPENAI_PROXY_URL
    • OPENAI_TIMEOUT_MS
    • OPENAI_REASONING_EFFORT
    • OPENAI_TRANSCRIPTION_MODEL

2026-07-20 - Milestone 2: Conversation import and GPT-5.6 structured analysis

  • Goal:
    • Accept real pasted AI conversations.
    • Analyze them through a server-side OpenAI Responses API call.
    • Validate structured output before returning it to the browser.
    • Show only three natural discussion questions in the initial UI.
  • Planned OpenAI API / model usage for this milestone:
    • API: OpenAI Responses API
    • Model: gpt-5.6
    • Use: imported conversation analysis with hidden hypotheses and public discussion-question generation
  • API endpoint implemented:
    • POST /api/analyze-conversation
  • Prompt version:
    • conversation-analysis-v1
  • Major files actually modified for Milestone 2:
    • .env.example
    • package.json
    • package-lock.json
    • scripts/diagnose-openai.ts
    • scripts/smoke-analyze.ts
    • server/index.ts
    • server/app.ts
    • server/analyze-conversation.ts
    • server/openai-runtime.ts
    • server/prompts/conversation-analysis-v1.ts
    • src/App.tsx
    • src/components/BuildWeekStatus.tsx
    • src/components/ChooseQuestion.tsx
    • src/components/ImportConversation.tsx
    • src/components/LandingPage.tsx
    • src/lib/prototype-data.ts
    • src/types.ts
    • src/vite-env.d.ts
    • vite.config.ts
    • tests/api-analyze-conversation.test.ts
    • tests/build-output.test.ts
    • tests/prototype-data.test.ts
  • Technical decisions:
    • Keep the API key on the server because any Vite frontend secret would be shipped to the browser.
    • Use one shared Zod schema for server validation and frontend-safe projection.
    • Keep hypotheses private and strip them before rendering initial discussion questions.
    • Treat demo analysis as a clearly labeled fixture, not as a live API success.
  • Commands run:
    • npm install openai zod concurrently supertest @types/supertest
    • npm install undici
    • npm run typecheck
    • npm run test
    • npm run lint
    • npm run build
    • npm run diagnose:openai
    • OPENAI_PROXY_URL=http://127.0.0.1:7890 npm run diagnose:openai
  • Real results so far:
    • Dependency install succeeded.
    • npm install undici succeeded.
    • npm run typecheck passed.
    • npm run test passed with:
    • # tests 13
    • # pass 13
    • # fail 0
    • # duration_ms 2288.924667
    • npm run lint passed.
    • npm run build passed:
    • dist/index.html 0.41 kB │ gzip: 0.28 kB
    • dist/assets/index-C4hcqiD9.css 40.62 kB │ gzip: 7.26 kB
    • dist/assets/index-BbZXgztQ.js 437.88 kB │ gzip: 134.24 kB
    • ✓ built in 2.21s
    • Explicit real smoke-test command created:
    • npm run smoke:analyze
    • Live OpenAI diagnostics executed with real requests:
    • Without explicit proxy, npm run diagnose:openai showed:
      • openaiProxyPresent: false
      • curl GET /v1/models succeeded
      • Node native fetch failed at about 10.5s with TypeError and cause code UND_ERR_CONNECT_TIMEOUT
      • OpenAI SDK minimal request failed at about 10.5s with APIConnectionTimeoutError
      • Failures reproduced at configured SDK timeouts 15000ms, 45000ms, and 120000ms
    • Local proxy discovery:
      • lsof -nP -iTCP -sTCP:LISTEN showed ClashX listening on 127.0.0.1:7890
    • With explicit proxy, OPENAI_PROXY_URL=http://127.0.0.1:7890 npm run diagnose:openai showed:
      • openaiProxyPresent: true
      • openaiProxyHost: 127.0.0.1:7890
      • curl GET /v1/models succeeded
      • curl POST /v1/responses succeeded
      • Node native fetch reached OpenAI and returned 401
      • OpenAI SDK requests reached OpenAI and returned AuthenticationError / invalid_api_key
    • Environment loading verification:
      • Shell inspection showed an existing exported OPENAI_API_KEY in the terminal environment
      • dotenv.config({ override: true }) was added to force server and diagnostic scripts to prefer .env over older exported shell values
    • .env loading verification after the override change:
      • 10 variables were read from .env
      • OPENAI_PROXY_URL was present
      • OPENAI_REASONING_EFFORT parsed as low
    • Live API validation progressed to parameter validation:
      • OpenAI returned 400 Unsupported value: 'minimal' is not supported with the 'gpt-5.6' model
      • Default fallback was updated from minimal to low
    • Two real test cases explicitly recorded:
    • Test case 1:
      • Input type: Node native fetch calling minimal POST /v1/responses with input Return exactly: connection_ok
      • Result without explicit proxy: failed in about 10.5s
      • HTTP status: none returned
      • Error class: TypeError
      • Error cause code: UND_ERR_CONNECT_TIMEOUT
    • Test case 2:
      • Input type: OpenAI SDK responses.create(...) calling full Milestone 2 schema with conversation-analysis-v1
      • Result without explicit proxy: failed in about 10.5s
      • HTTP status: none returned
      • Error class: APIConnectionTimeoutError
      • Result with explicit proxy and live upstream reachability: request reached OpenAI and returned upstream validation / auth errors (401 invalid_api_key, later 400 invalid reasoning effort) instead of local connect timeout
  • Known limitations while Milestone 2 is in progress:
    • No successful end-to-end live conversation-analysis response has been captured yet in this evidence log.
    • Demo analysis remains fixture-backed and does not exercise the live API path.
    • The Milestone 1 voice / tree components still exist in the repository but are not part of the Milestone 2 runtime flow.
    • Current server behavior still depends on a valid OpenAI key plus a working local proxy path in this machine environment.
    • npm run diagnose:openai required network permission approval in this environment; one later rerun was blocked when approval was not granted.

2026-07-20 - Milestone 3: First spoken response and initial Expression Tree

  • Goal:
    • Let the user select one real GPT-generated discussion question.
    • Record a real first spoken response in the browser.
    • Send audio to the server for transcription.
    • Let the user review or lightly edit the transcript.
    • Analyze the confirmed transcript with GPT-5.6 into an initial three-layer Expression Tree.
  • API endpoints implemented:
    • POST /api/transcribe-first-response
    • POST /api/analyze-first-response
  • Actual model usage in this milestone:
    • gpt-4o-mini-transcribe
    • Use: first-response audio transcription on the server
    • gpt-5.6
    • Use: transcript-to-expression-tree analysis on the server
  • Prompt version:
    • first-response-tree-v1
  • Structured output schema implemented for first-response analysis:
    • eventNodes[]
    • id
    • text
    • evidence
    • meaningNodes[]
    • id
    • text
    • evidence
    • relatedEventIds[]
    • emotionHypotheses[]
    • id
    • internalLabel
    • description
    • evidence[]
    • confidence
    • relatedMeaningIds[]
    • broadExpressions[]
    • originalText
    • reasonItIsBroad
  • Major files actually modified for Milestone 3:
    • docs/architecture.md
    • docs/build-week-evidence.md
    • package.json
    • scripts/smoke-first-response.ts
    • scripts/smoke-transcribe.ts
    • server/analyze-first-response.ts
    • server/app.ts
    • server/prompts/first-response-tree-v1.ts
    • server/transcribe-first-response.ts
    • src/App.tsx
    • src/components/ChooseQuestion.tsx
    • src/components/ExpressionTree.tsx
    • src/components/SpeakPage.tsx
    • src/session-state.ts
    • src/types.ts
    • tests/first-response-flow.test.ts
  • Technical decisions:
    • Use browser MediaRecorder for real capture instead of reusing Milestone 1 visual-only simulation.
    • Keep transcription server-side so no API credential ever reaches the browser.
    • Store transcript drafts with a userModified flag so ASR output and user edits stay distinguishable.
    • Validate and additionally ground event, meaning, emotion-evidence, and broad-expression spans against transcript text before accepting tree output.
    • Keep emotionHypotheses private in the UI and render only a soft hidden top layer because emotional naming is collaborative and not finalized in this milestone.
  • Commands run:
    • npm run lint
    • npm run typecheck
    • npm run test
    • npm run build
    • npm run smoke:first-response
    • npm run smoke:transcribe
  • Real results:
    • npm run lint passed.
    • npm run typecheck passed.
    • npm run test passed with:
    • # tests 28
    • # pass 28
    • # fail 0
    • # duration_ms 2213.879375
    • npm run build passed:
    • dist/index.html 0.41 kB │ gzip: 0.28 kB
    • dist/assets/index-By343B5P.css 41.13 kB │ gzip: 7.41 kB
    • dist/assets/index-BQobVXpq.js 463.94 kB │ gzip: 139.61 kB
    • ✓ built in 2.26s
    • npm run smoke:first-response succeeded with a real upstream request:
    • Input type: short fixed English transcript string, not demo fixture
    • Result: success
    • Source: live
    • Model: gpt-5.6
    • Prompt version: first-response-tree-v1
    • Event nodes: 2
    • Meaning nodes: 2
    • Emotion hypotheses: 3
    • Broad expressions: 0
    • Real request latency: 11074ms
    • OpenAI response ID: resp_0eaf6b40444281f7006a5dd1b1994c819a9496b7f3b1030723
    • OpenAI request ID: req_6664156d0c9e44d7a2b3bce04961ca98
    • Client request ID: a709694a-8199-4bf2-b8f6-577485dc167c
    • npm run smoke:transcribe reached the real upstream transcription API but failed:
    • Input type: locally generated .wav sample file
    • Result: failure
    • HTTP status: 400
    • Error class: BadRequestError
    • Error code: invalid_value
    • Error message: Audio file might be corrupted or unsupported
    • Real request latency: 1124ms
    • OpenAI request ID: req_015f8c5d0da5436981afd8c7339c7971
    • Conclusion: the failed smoke test does not prove a code-path failure; it proves the local sample file used for the smoke test was not accepted by the upstream transcription API.
  • Two explicitly recorded real test inputs and results:
    • Test case 1:
    • Input type: short transcript text passed to POST /api/analyze-first-response
    • Result: real GPT-5.6 structured analysis succeeded and validated
    • Latency: 11074ms
    • Test case 2:
    • Input type: local audio upload passed to POST /api/transcribe-first-response
    • Result: request reached OpenAI but returned 400 invalid_value because the sample audio was corrupted or unsupported
    • Latency: 1124ms
  • Known limitations after Milestone 3:
    • No successful real browser-recorded transcription smoke test has been captured in this evidence log yet.
    • The automated transcription smoke test currently depends on generating a valid local audio sample; the first generated sample was not accepted by the upstream API.
    • Expression Tree animation is state-driven after transcript analysis completes; it is not yet streaming node growth in real time while the user speaks.
    • Emotion hypotheses are intentionally hidden and not yet confirmable in the UI; that work is deferred to the next milestone.

Milestone 2 Schema

  • Shared validation location:
    • src/types.ts
  • Structured output schema shape:
    • conversationSummary: string
    • explorationHypotheses: Array<{ id: string; internalLabel: string; description: string; evidence: string[]; confidence: number }>
    • discussionQuestions: Array<{ id: string; hypothesisId: string; question: string; sourceContext: string }>
  • Structured result fields:
    • conversationSummary
    • explorationHypotheses
    • discussionQuestions
  • Enforced invariants:
    • exactly 3 hypotheses
    • exactly 3 discussion questions
    • discussion questions must be distinct
    • discussion questions must reference valid hypothesisId values
    • discussion questions must end with ?
    • deterministic or diagnostic question phrasing is rejected

Known Gaps Before Submission

  • No successful end-to-end live conversation-analysis response has been recorded yet.
  • No semantic-transfer evaluation, acoustic scoring, or user-study export exists yet.
  • No deployment walkthrough has been documented beyond .env.example and the local server boundary.

Challenges we ran into

Helping the learner clarify meaning without speaking for them

A language model can easily generate an impressive sentence that does not actually belong to the learner.

The central design challenge was preventing Cosu from becoming another AI rewriting tool.

Cosu therefore asks clarification questions before generating the final expression. The learner must be able to reject, correct, or refine the model’s interpretation.

The product principle is:

Cosu does not put better words into your mouth. It helps you find the words for what you already mean.

Distinguishing precision from complexity

A longer sentence or rarer word is not automatically more precise.

Sometimes the most accurate improvement is a simple word such as relieved. In other situations, precision requires expressing a relationship between multiple feelings, such as relief mixed with uncertainty.

Cosu therefore evaluates semantic fit rather than vocabulary difficulty alone.

Asking the right amount of questions

Too few questions lead to shallow rewriting. Too many questions make the interaction feel like an interview and interrupt speaking practice.

I had to design the system to ask only the question that creates the most useful distinction.

Verifying ownership instead of repetition

A learner may repeat a model sentence perfectly without being able to use it independently.

To address this, Cosu changes the scenario and evaluates whether the learner can reconstruct the underlying meaning in their own words.

This transfer step is central to the product.

Integrating semantic and acoustic evaluation

A response can be semantically precise but difficult to deliver, or fluent but semantically vague.

Combining these two forms of evaluation without collapsing them into one unclear score was another important design challenge.

Maintaining a clear educational boundary

Precise emotional language may support reflection, but Cosu does not diagnose users, provide treatment, or present itself as a therapist.

Its role is to help learners develop the English required to describe their experiences and ideas more clearly.

Accomplishments that we're proud of

We are proud that Cosu moves beyond both traditional vocabulary learning and one-click AI rewriting. Instead of simply recommending better phrases, Cosu helps the learner work through a full meaning-refinement loop: starting from a prior AI conversation, identifying what they were trying to express, clarifying the meaning underneath the first answer, selecting language they can genuinely own, and then speaking again with greater precision. The current Build Week MVP creates this complete learning loop: imported conversation -> discussion angle selected -> first spoken response captured -> intended meaning clarified -> target expressions generated -> second response practised -> expression saved for reflection

We are also proud that Cosu combines structured semantic reasoning with spoken practice, rather than treating speaking as a text-only rewriting task. GPT-5.6 is used to analyze the imported conversation, interpret the learner’s first spoken response, refine intended meaning during the Clarify stage, and generate candidate expressions for the Own It stage. This means the product is not only helping the learner produce a sentence, but helping them move closer to saying what they actually mean. Most importantly, Cosu does not treat the learner as a passive recipient of AI-generated language. The learner remains responsible for choosing the discussion path, reviewing or correcting the transcript, confirming or adjusting the clarified meaning, selecting expressions they can actually use, and finally speaking again in their own words. Verified accomplishments in the current MVP include: 1 complete end-to-end demo session run and verified during the final submission audit a working multi-step flow from Start to Done real server-backed OpenAI integration for conversation analysis, first-response analysis, clarify-meaning generation, and target-expression generation transcript review before downstream reasoning, so later steps are grounded in learner-confirmed language an expression-tree style meaning view connecting what happened, what it meant, and the deeper emotional hypothesis a saved-expression step at the end of the session, allowing the learner to leave with a personally selected reflection word or phrase an audited engineering baseline including lint, typecheck, tests, production build, architecture notes, and build evidence documentation One strong before-and-after example from the current MVP is: Before: “I don't like it when people from my past ask about me. I think maybe they want to know whether I failed. It makes me feel bad and I don't want to answer.” After: “To me, it means I hear their concern as a quiet test of whether my life is going well enough. That is why I start to feel self-conscious.”

What we learned

Building Cosu changed how I think about language learning.

Fluency is not only the ability to produce sentences quickly. It also depends on whether a person can identify what they mean with enough precision to express it.

I learned that personalization should mean more than adjusting vocabulary difficulty. The most memorable language is often language connected to something the learner has genuinely tried to understand or communicate.

I also learned that an effective AI tutor should not always answer immediately.

Sometimes the most useful teaching action is to ask the one question that helps the learner notice a distinction they could not previously express.

Finally, I learned that the strongest role for AI in this product is not to replace the learner’s voice. It is to help the learner develop one.

What's next for Cosu

The next step is to test Cosu across personal, academic, and professional speaking situations.

I plan to evaluate:

  • whether learners feel that the final expression accurately represents their intended meaning;
  • whether they can reuse the meaning in a new context without seeing the model sentence;
  • whether repeated practice improves semantic precision and spoken fluency;
  • how Personal Expression Memory develops over time;
  • whether Cosu can identify recurring patterns in how an individual learner struggles to express ideas.

Future versions could also allow users to bring in useful language from previous AI conversations, articles, meetings, or personal notes and transform it into speaking practice grounded in their own context.

The long-term vision is a speaking agent that does not merely correct what users say.

It helps them discover what they mean, find the language that fits, and make that language part of their own voice.

Built With

Share this project:

Updates