VANTA Study

• 

  Lecture transcript and playback

• 

  Explicit consent and GPT-5.6 live controls

• 

  GPT-5.6-generated recall question

• 

  Partial evaluation with hints, evidence, and timestamps

• 

  Improved answer with a complete score

Inspiration

Hello, my name is Phillip, and in a few months I will begin studying computer science in Germany.

With VANTA Study, I simply wanted to build a tutor that I can use at any time and that makes learning for all of my classes easier.

Most lecture tools stop after recording or transcription. They help students store information, but they do not necessarily help them understand it, remember it, or return to the exact explanation when something is unclear.

That became the idea behind VANTA Study: turning lectures into searchable, verifiable, and active learning experiences.

VANTA Study is part of my larger personal assistant project, VANTA.OS, but the Study workflow was built as the main focus of the OpenAI Build Week.

What it does

VANTA Study can record a lecture or import an existing audio file.

The audio is processed locally and becomes a study session with:

• a locally generated transcript
• synchronized audio playback
• full-text search
• clickable transcript timestamps
• direct navigation back to the original explanation
• evidence-grounded recall questions
• structured answer evaluation with full, partial, and missing coverage
• locally calculated scores and verdicts
• a two-level hint system before revealing the solution

The central idea is that the learning process always stays connected to the original lecture.

When a student gives an incomplete or incorrect answer, VANTA does more than display a score. It provides hints and links the evaluation back to relevant transcript segments. The student can click a timestamp, listen to the original explanation again, improve the answer, and retry.

How I built it

VANTA Study is a desktop application built with React, TypeScript, Tauri, Rust, SQLite, and Python-based local transcription.

The architecture is local-first:

• lecture audio remains on the computer
• transcription runs locally
• sessions and transcripts are stored locally
• transcript search uses SQLite FTS5
• playback and timestamp navigation remain local
• cloud AI is optional and disabled by default

GPT-5.6 Sol is used only after explicit user permission.

For the live recall flow, only relevant transcript segments are sent to the model. GPT-5.6 generates questions and returns structured semantic assessments for each expected key point, together with feedback, a solution summary, and grounded evidence segment IDs.

VANTA then controls the final learning experience:

• Rust validates the structured response
• unknown or ungrounded evidence IDs are rejected
• score and verdict are calculated locally and deterministically
• the two-level hint ladder is generated locally
• the solution reveal order is controlled locally
• timestamps remain connected to the original lecture

This prevents the model from becoming the final authority over scoring, evidence, or pedagogy.

How I used Codex

Codex was my primary implementation agent during the Build Week.

English is not my first language, so I used AI to help me correct the grammar and organize some of the technical information in this submission. The ideas, decisions, tests, and personal parts are still my own.

I defined the product direction, feature scope, privacy boundaries, architecture requirements, and real-world test scenarios. Codex helped translate those decisions into working React, TypeScript, Rust, SQLite, and test code.

My development workflow was:

1. Define a small and testable milestone.
2. Let Codex implement it.
3. Run automated tests and builds.
4. Review the implementation independently.
5. Fix any findings.
6. Perform the real application test myself.
7. Commit only after the milestone was verified.

This workflow helped me move quickly while keeping the project understandable and testable.

Challenges

Building a complete local lecture pipeline

Recording audio was only the first step. The project also needed reliable session storage, transcription jobs, transcript segments, search indexing, synchronized playback, timestamp seeking, recovery after interruption, and media import.

Each part had to connect to the same study session without creating separate pipelines for recordings and imported files.

Keeping GPT output grounded

A model-generated answer alone is not enough for a reliable study tool.

Every evidence ID returned by GPT-5.6 is checked against the transcript segments that were actually included in the knowledge pack. If the model references an unknown segment, the evaluation is rejected.

Making the hint system reliable

During live testing, model-generated hints could reveal too much of a missing answer. Originally, this caused the complete evaluation to be rejected.

I changed the architecture so GPT-5.6 remains responsible for the semantic assessment, while VANTA always creates both hints locally and deterministically.

This made the live GPT workflow as reliable as the local demo flow without requiring another API call.

Keeping cloud usage controlled

The live GPT path requires:

• an explicit backend flag
• a backend API key
• the allowed GPT-5.6 Sol model
• visible user consent
• a deliberate user action

A persistent local ledger reserves every request before it is sent. It enforces a hard limit of 100 live calls and an internal default budget of $8, with no automatic retries or background calls.

Accomplishments that I am proud of

What I am most proud of is the simple fact that I am nowhere near being a professional developer, but with the right tools, I can still build almost anything that makes my life easier.

• A complete lecture-to-recall workflow built during the Build Week
• Local transcription, storage, search, playback, and timestamp navigation
• Real GPT-5.6 question generation and answer evaluation
• Deterministic local scoring and hint control
• Evidence-linked evaluations with direct jumps back to the lecture
• Recovery and persistence across application restarts
• A provider-agnostic architecture instead of a simple GPT wrapper
• 139 passing frontend tests and 165 passing Rust tests in the private submission repository
• A working product that I genuinely plan to use during my own studies

What I learned

The biggest lesson was that building with AI is not only about writing prompts.

I also learned that even as an absolute beginner in this completely new field, I can sit in front of my computer and build tools that actually help me in real life. This is by far the best thing I have learned in 2026, and I am sure that even more great things will come with this new chapter of my life.

The difficult part is defining clear boundaries:

• What should the model decide?
• What must remain deterministic?
• What data may leave the device?
• What should happen when model output is structurally valid but pedagogically unsafe?
• How can AI-generated code be reviewed and tested independently?

I also learned how important it is to separate implementation from review. Using one agent to implement and another independent reviewer helped catch integration problems that a single model could easily miss.

What's next

The current version proves the complete core workflow.

Next, I want to expand VANTA Study with:

• a full flashcard and spaced-repetition system
• long-term learning history
• lecture and subject organization
• improved recall sessions
• mobile access
• timetable-aware lecture routing
• better study progress visualization

VANTA Study began as the system I wanted for my own computer science degree.

For me, VANTA is a system that I will always keep upgrading, because I want it to be shaped around my life and make things as easy and automated as possible.

For example, I want VANTA to know when my volleyball training starts, start a timer that tells me when I need to head out, and automatically create a small to-do list with everything I need, so I do not forget anything.

The OpenAI Build Week was actually not part of my plan when I started building VANTA, but it gave me an extremely valuable opportunity to let experienced judges review my project.

You do not get an opportunity like that very often, especially as someone like me who is still completely new to computer science.

Built With

• ai
• api
• codex
• faster-whisper
• fts5
• gpt-5.6
• local-first
• openai
• python
• react
• rust
• speech-to-text
• sqlite
• tauri
• typescript
• vite
• vitest