Socratix

Socratix is a memory-enabled, agentic Socratic study companion that unifies two flows normally kept separate: AI-assisted course scaffolding (instructor side) and Socratic tutoring (student side).

Comment

Inspiration

Socratix started from a problem we kept seeing in online courses: students can watch a lesson and still carry a hidden misconception into the next topic. Multiple choice can show that a student is wrong, but it usually does not show why. We wanted to build a tutoring layer that helps students expose their reasoning, correct misconceptions Socratically, and give instructors a clear signal about what the class is actually struggling with.

We chose introductory inferential statistics because the misconceptions are concrete and high-impact. A student saying “p = 0.03 means there is a 3% chance the null hypothesis is true” is not just slightly wrong; it reveals a specific reasoning error that a good tutor should catch.

What it does

Socratix is a Socratic study companion built on top of an existing LMS. Students learn inside a normal course lesson with video and written content, then answer free-text questions underneath the lesson.

The tutoring layer:

  • diagnoses student free-text attempts using GPT-5.6 structured outputs
  • classifies attempts as CORRECT, SLIP, CONCEPTUAL, or PREREQUISITE_GAP
  • asks Socratic follow-up questions instead of giving away answers
  • refuses answer-extraction attempts like “just tell me the answer”
  • keeps tutoring inside the instructor-approved syllabus and method
  • generates next-item practice and marks it for instructor review
  • logs next-item correctness to measure whether learning transferred
  • stores per-student concept memory
  • gives instructors a dashboard of misconceptions, concept performance, pending generated practice, and recent attempts

For the demo, the course is introductory inferential statistics, focused on sampling distributions, standard error, confidence intervals, and p-value interpretation.

How we built it

The base LMS already had courses, chapters, lessons, enrollment, authentication, payments, and dashboards. During the hackathon, we used Codex to extend it with the Socratix tutoring layer.

We added Prisma models for Concept, Question, Attempt, and per-student concept memory. Questions are attached to existing LMS lessons, and attempts store the student response, diagnosis JSON, correctness, and whether the attempt was a next item.

The AI layer uses the OpenAI Responses API with GPT-5.6 and Zod structured outputs. GPT-5.6 returns a typed classify_attempt object, and our application code controls the branching. The model does not run as a free autonomous agent; it diagnoses and proposes the tutoring move, then the app decides what happens next.

We also built guardrails in application code. If a student tries to get the answer directly, Socratix forces a Socratic question. If the student moves outside the syllabus or uses a method that conflicts with the instructor’s method note, the tutor redirects them.

Codex helped us move quickly through schema design, migrations, UI wiring, server actions, OpenAI integration, guardrails, seed data, the tutor dashboard, and repeated build/typecheck verification.

Challenges we ran into

The hardest product challenge was scope. The existing LMS is already broad, but the hackathon story needed to be about the new AI tutoring layer, not course management or payments. We had to cut anything that did not make the Socratic loop stronger.

The second challenge was making the demo educationally honest. It would have been easier to claim that the AI watches a YouTube video and creates a whole course. Instead, we kept the source of truth as instructor-approved lesson context and questions. The video is supporting material, not the authority.

The third challenge was reliability. Live model calls can be more generous or word responses differently than a recorded script expects. We solved this by combining GPT-5.6 structured diagnosis with deterministic guardrails and demo-safe fallbacks, so the product remains stable while still using GPT-5.6 for the core AI path.

Accomplishments that we're proud of

We built an end-to-end tutoring loop inside a real LMS rather than a standalone chatbot demo.

A student can submit a free-text answer, receive a Socratic follow-up, correct their reasoning, attempt a next item unaided, and have that next-item correctness logged. The instructor can then open a tutor dashboard and see misconception telemetry, concept performance, generated practice awaiting review, and recent attempts.

We are also proud that the design keeps the instructor in control. Generated practice is marked for review, the tutor respects the syllabus, and the system reports learning signals back to the teacher instead of replacing them.

What we learned

We learned that “AI tutor” is too vague unless the product has a measurable learning loop. Next-item correctness became the anchor metric because it asks a more useful question: can the student solve the next problem without help?

We also learned that Socratic behavior cannot rely only on a system prompt. Models want to be helpful, and students can pressure them into giving answers. Guardrails need to exist in application code.

Finally, we learned that Codex is especially strong when extending an existing codebase. It helped us read local patterns, add Prisma migrations, wire server actions, and keep the changes scoped to the new Socratix layer without rebuilding the LMS.

What's next for Socratix

Next, we want to deepen the instructor workflow: content approval queues, editable concept maps, and richer misconception analytics by cohort.

We also want to improve memory over multiple sessions so Socratix can personalize follow-up based on what a student has struggled with before.

Longer term, we would explore transcript-assisted course setup, but still with instructor approval as the source of truth. The goal is not to let AI invent a course from a video; it is to help instructors turn their own material into a tutoring loop that catches misconceptions and measures transfer.

Built With

Share this project:

Updates