StudyFlow

StudyFlow is an adaptive lecture player: it transcribes your video, scores how dense each section is, and speeds up or slows down playback automatically.

Comment

Inspiration

Lecture videos aren't uniformly hard. The first five minutes are usually announcements and review; then there's a stretch where three new ideas land on top of each other in ninety seconds. Every video player treats those two moments identically - you pick one speed and live with it, either wasting time on the easy parts or getting steamrolled by the dense ones. I wanted a player that noticed the difference itself and adjusted, the way a good reader slows down at the hard paragraph without being told to.

What it does

Upload a lecture video. StudyFlow transcribes it, scores how conceptually dense each section is on a 1–5 scale with a confidence value, and turns that into a playback-rate profile - faster through familiar material, back to normal speed for anything dense. The model only ever estimates complexity; the application decides the actual rate:

$$\text{rate} = \min\big(\text{rate}{\max},\ \max(\text{rate}{\min},\text{rate}_{\text{complexity}})\big)$$

The reasoning behind every speed change is visible in the player, not hidden behind a black box - plus a private account, a lecture library, and a fully custom player with captions, skip, and keyboard shortcuts.

Built with

Python, FastAPI, SQLAlchemy, Alembic, Pydantic, Celery, Redis, PostgreSQL/SQLite, FFmpeg/FFprobe, OpenAI API (transcription + structured-output classification), Next.js, React, TypeScript, TanStack Query, Vitest, Playwright, pytest.

How I built it

FastAPI backend, Next.js frontend, SQLAlchemy/Postgres, Celery/Redis for background work, OpenAI for transcription and structured-output complexity classification. The pipeline is upload → extract/chunk audio → transcribe → classify → smooth into a playback profile → play. Every external dependency (transcription, classification, job dispatch) sits behind a small protocol interface, so the whole thing runs end-to-end in a mock mode with zero API keys - the same code path real providers use, just swapped in via config.

Around that core, a real product grew: accounts with sliding sessions, a private lecture library with retry/delete, chunked uploads for large files, duplicate detection by content hash so re-uploading the same lecture doesn't cost another transcription pass, signed short-lived video URLs instead of an open endpoint, and a fully custom video control bar built from scratch.

Challenges I ran into

Transcription APIs cap request duration. A 40-minute lecture blew past the model's ~23-minute limit per request. The fix was splitting the audio into chunks with ffmpeg and stitching the transcripts back into one ordered timeline, offsetting each chunk's timestamps by its position in the full lecture.

Smoothing the playback profile was harder than the core feature. Raw per-segment complexity scores are noisy, so segments needed smoothing - but the smoothing logic regressed three separate times: first too jittery, then over-corrected and erased a genuinely important dense segment, then over-corrected again and collapsed almost the entire lecture to 1×. The fix wasn't another heuristic; it was a hard invariant - smoothing may only ever make a segment more conservative than the raw classification said, never faster - plus rolling back an overgrown pipeline to something simple enough to reason about by hand.

The browser's native video controls can't be extended. Wanting custom skip/caption/fullscreen buttons in the same bar as play/pause meant building the whole control bar from scratch. That surfaced a second trap: requestFullscreen() only fullscreens the exact element it's called on, so calling it on the <video> itself left every custom control and the pace-timeline behind on exit from view. The fix was requesting fullscreen on the wrapping container instead.

Large uploads broke on infrastructure I hadn't looked at closely. Routing a big video file through Next.js's dev proxy layer hit a silent 10 MB body cap and then a connection reset - nothing about the FastAPI backend was wrong, the file just never really got there. Uploading straight to the API origin, then adding real chunked upload, fixed it properly instead of papering over it.

Accomplishments that I'm proud of

Getting a genuinely working, end-to-end adaptive system running in mock mode with zero credentials, then swapping in real OpenAI transcription and classification without touching a single call site - the protocol boundary held up exactly as intended. Catching a real class of regression (the smoothing ratchet bug) by diagnosing the actual mechanism instead of patching symptoms, and enforcing it with a hard invariant test. Shipping things that go past hackathon-MVP scope because they turned out to matter in practice: duplicate detection, signed video URLs, chunked resumable-feeling uploads, and a fullscreen-aware custom player built entirely from scratch.

What I learned

Treat "the model estimates, the application decides" as a hard architectural line, not a suggestion - it's what made every regression here debuggable instead of mysterious. And build the thing, then actually use it: almost every real bug (the duration cap, the fullscreen behavior, the upload proxy limit) only showed up once a real video, at real length, went through the real flow - not from reading the code.

What's next for StudyFlow

Topic-aware segmentation instead of fixed time windows, so section boundaries line up with where a new idea actually starts. A feedback loop that learns from when I manually override the adaptive speed, to make the classifier's judgment better calibrated to that specific viewer over time. Support for a local/offline transcription model behind the existing provider interface, for lectures that shouldn't leave a laptop. And a real production deployment - Postgres and Celery in the cloud instead of local dev - to see how it holds up outside a single machine.

Built With

Share this project:

Updates