Socratic Runtime

Inspiration

Most AI coding assistants optimize for immediate task completion. Socratic Runtime optimizes for learning.

I am a university student interested in neuroscience, education, and human learning. I have seen students respond to AI coding tools in two very different ways: some avoid them because they are afraid of losing the opportunity to learn independently, while others generate complete answers before attempting the problem themselves.

I wanted to explore a third possibility: an AI tutor that supports learning without removing the cognitive work.

My hypothesis is that students learn programming more effectively when they have space to struggle productively—to retrieve knowledge, develop an approach, make mistakes, and revise their reasoning—before receiving help. This idea is informed by concepts from learning science and neuroscience, including active retrieval, productive struggle, feedback timing, and cognitive control.

Socratic Runtime translates that hypothesis into an interaction inside the learner’s editor. It stays quiet while independent reasoning remains productive and asks one focused question when help would be useful.

It does not claim to have experimentally proven an educational or neuroscientific effect. Instead, it is a working prototype for exploring what an AI tutor built around restraint might feel like.

What it does

Socratic Runtime is an ambient VS Code programming tutor that knows when to stay silent.

The learner opens an ordinary programming file and starts a session with one command. The task can come from:

• an @socratic-task comment;
• selected text;
• or, when no task is provided, a task inferred from the current file and confirmed or edited by the learner.

From there, the learner simply continues coding.

After a meaningful revision and a short pause, GPT-5.6 Luna considers the task, the previous and current code, editor diagnostics, and the learner’s recent trajectory. It then chooses one of three actions:

1. Remain silent while experimentation, progress, or self-correction appears productive.
2. Ask one focused Socratic question when the learner appears genuinely stuck.
3. Consider the task complete, end the active session, and provide a reference approach for reflection.

During an active task, Socratic Runtime does not generate code, reveal a complete solution, or provide a mechanical sequence of edits. It asks only one concise question at a time.

Questions can recur if the learner overcomes one obstacle, makes progress, and later becomes stuck on a different issue. The tutor evaluates the overall trajectory rather than relying on a fixed intervention count.

The learner can also request a nudge, pause or resume the session, inspect the Luna decision trace, or end the session at any time.

Socratic Runtime never edits or executes the learner’s code.

The interaction is language-neutral. Python is used for the demonstration, but no Python-specific runtime, test framework, exercise manifest, or project configuration is required.

It also reuses the learner’s existing ChatGPT/Codex sign-in. There is no separate product account or API key to configure.

One extension. One existing sign-in. One command on the file the learner is already editing.

How I built it

I was the sole human builder and product architect. I used Codex as my primary engineering partner throughout OpenAI Build Week.

Socratic Runtime is a TypeScript and Node.js extension for Visual Studio Code. It connects to GPT-5.6 Luna through the authenticated Codex CLI, reusing the learner’s existing ChatGPT/Codex authentication.

The extension observes meaningful revisions to the active file. After typing stops for two seconds, it creates a bounded learning-state packet containing:

• the programming task;
• the previous and current code;
• a compact revision diff;
• editor diagnostics;
• the recent sequence of decisions and interventions;
• and whether the learner explicitly requested help.

GPT-5.6 Luna medium owns the semantic and pedagogical judgment. It evaluates the learner’s trajectory rather than reacting to a single syntax error, elapsed-time threshold, or fixed failure count.

The deterministic extension host owns the runtime boundaries:

• cancellation of stale assessments;
• strict structured-output schemas;
• bounded input and output sizes;
• process isolation and timeouts;
• environment and data minimization;
• and active-session solution-leakage protection.

Model output is validated before it reaches the learner. If a proposed question contains code, hidden expected outputs, multiple questions, or a mechanical edit recipe, the intervention is blocked and the runtime fails safely.

Codex helped me explore and revise the architecture, implement and refactor the TypeScript extension, design tests, review privacy and security boundaries, refine the interface, write documentation, audit dependencies, and prepare the installable VSIX release.

Codex therefore served two different roles in the project: it was my development partner while building the extension, and the authenticated Codex CLI now acts as the runtime bridge to GPT-5.6 Luna.

Challenges I ran into

Finding the right intervention timing

The hardest problem was deciding when the tutor should interrupt.

An early version placed deterministic gates, failure counts, confidence thresholds, and executable verification around the model. The architecture appeared rigorous, but beginner-style testing exposed a fundamental problem: the tutor remained silent even when the learner was visibly struggling.

The system was technically functioning but educationally ineffective.

That failure led to the most important architectural decision in the project. I removed verifier-driven pedagogical gates and gave GPT-5.6 Luna responsibility for interpreting the learner’s trajectory.

Deterministic code still owns safety, cancellation, schemas, privacy, and process execution. It no longer attempts to encode human learning as a collection of numerical thresholds.

Distinguishing thinking from being stuck

A learner may stop typing because they are thinking. Elapsed time alone is therefore not evidence of confusion.

Socratic Runtime does not interrupt simply because the learner has been inactive. Assessments are triggered by meaningful revisions, and the system considers how the code and recent trajectory are changing.

Preventing solution leakage

Even a question can reveal too much.

A tutor might avoid giving a complete answer while still including code, pseudocode, hidden expected outputs, or a disguised sequence of edits. Socratic Runtime therefore validates every active-session intervention against a conservative safety boundary.

The goal is not merely to avoid revealing the final answer. It is to preserve the learner’s responsibility for constructing the solution.

Keeping the product easy to try

Educational tools often require accounts, API credentials, exercise manifests, test suites, or course configuration before the learner can begin.

I wanted the opposite experience.

The final workflow is:

1. Install the extension.
2. Use an existing ChatGPT/Codex sign-in.
3. Open a programming file.
4. Start Socratic Runtime.
5. Keep coding.

Accomplishments that I’m proud of

I am proudest of turning an unusual educational principle into a complete, installable product:

For an AI tutor, restraint can be a feature.

The v0.1.0 release includes:

• trajectory-aware silence, questioning, and completion;
• focused Socratic intervention without active-task solution generation;
• learner-controlled nudges, pausing, resuming, and session ending;
• automatic post-completion reference material;
• language-neutral operation on ordinary programming files;
• existing ChatGPT/Codex authentication with no separate API key;
• inspectable model decisions and latency through the Luna trace;
• bounded context and privacy-oriented data minimization;
• process isolation, cancellation, strict schemas, and leakage protection;
• automated TypeScript and VS Code Extension Host verification;
• an authenticated simulation of a complete beginner journey;
• and a public, MIT-licensed, installable VS Code extension.

The authenticated simulation exercises the complete runtime journey with the real model: intentional silence, Socratic intervention, completion, and post-task reference generation.

These engineering evaluations demonstrate that the product behaves as designed. They do not establish improved learning outcomes. That requires future evaluation with real students and educators.

What I learned

I learned that agentic coding is not simply asking an AI to build an application.

It requires product judgment, precise specifications, evaluation, iteration, and a willingness to discard architecture that does not serve the user.

Codex dramatically accelerated implementation, but the human role remained essential. I had to define the educational intention, decide what the tutor should deliberately refuse to do, recognize when technically defensible behavior felt wrong, and determine which responsibilities belonged to the model and which required deterministic enforcement.

The most important technical lesson was that probabilistic and deterministic components should be divided by responsibility.

GPT-5.6 Luna is well suited to interpreting semantic progress, uncertainty, alternative strategies, and the overall learning trajectory.

Deterministic code is better suited to schemas, privacy, cancellation, process isolation, output limits, and safety enforcement.

Trying to make deterministic thresholds decide whether a learner was genuinely stuck produced a system that appeared rigorous but behaved poorly. Giving the model semantic responsibility while preserving strict host boundaries resulted in a more coherent product.

More broadly, I learned that a responsible AI product is shaped not only by what it can do, but also by what it deliberately refuses to do.

What’s next

The next priority is evaluation with real students and educators.

I want to investigate whether:

• the timing of interventions feels helpful;
• intentional silence creates useful space rather than confusion;
• the questions preserve learner autonomy;
• different experience levels require different tutoring behavior;
• learners understand why the tutor sometimes remains quiet;
• and the interaction improves retention or independent problem-solving.

I also want to validate the extension across more programming languages, operating systems, editors, multi-file tasks, and authentic coursework.

A future version could optionally combine Luna’s source-level judgment with executable verification when a suitable language environment and test suite are available. The current prototype deliberately favors near-zero setup and language-neutral operation over claiming formally verified correctness.

Longer term, I want to explore this interaction beyond programming. Mathematics and other structured learning activities may also benefit from an assistant that observes a problem-solving trajectory, recognizes repeated struggle, and offers the smallest intervention necessary to restore productive thinking.

Socratic Runtime is an experiment in a different relationship with educational AI: one where the model does not merely know how to help, but also knows when to wait.

Built With

• artificial-intelligence
• codex
• codex-cli
• edtech
• education
• gpt-5-6
• node.js
• openai
• socratic-learning
• typescript
• visual-studio-code
• vscode-extension