PaperTrial

Defend your thesis before the real defense destroys you.

Comment

🎯 Inspiration

Every year, millions of graduate students walk into their thesis defense woefully unprepared. The statistics are brutal:

  • \( 70\% \) of PhD candidates report severe anxiety before their defense.
  • The average student gets less than 2 hours of mock practice before the real thing.
  • Friends give shallow feedback. Advisors are too busy. There is no tool that attacks your methodology the way a real committee does.

I asked myself: What if an AI could read your entire 100-page thesis, find every weak point, and then grill you on it — live, with voice, under real pressure?

That question became PaperTrial.

🛠️ How I Built It

PaperTrial is a full-stack distributed system built across 5 computation phases:

Phase 1 — Ingestion Pipeline

I use PyMuPDF to extract raw text from uploaded PDFs, preserving page-level metadata. The text is split into semantic chunks using LangChain's RecursiveCharacterTextSplitter with:

$$\text{chunk_size} = 500, \quad \text{overlap} = 50$$

Phase 2 — Vector Embedding (RAG Foundation)

Each chunk is embedded into a \( 768 \)-dimensional vector using Google Vertex AI (text-embedding-004) and stored in a PostgreSQL database with the pgvector extension. I use an HNSW index for sub-millisecond cosine similarity retrieval:

$$\text{similarity}(A, B) = \frac{A \cdot B}{|A| \cdot |B|}$$

Phase 3 — Weakness Detection

Gemini 2.5 Flash performs a multi-pass structural analysis on the vectorized chunks, hunting for sampling biases, missing citations, and logical fallacies. Gemini 2.0 Flash then synthesizes these into exactly 5 targeted attack questions.

Phase 4 — Live Audio Defense Engine

The student enters a real-time voice session:

  • 🎙️ Deepgram (WebSocket STT) transcribes speech in real-time
  • 🧠 The transcript is embedded and matched against the thesis via RAG
  • 📝 Gemini grades the answer for Accuracy, Depth, and Coherence
  • 🔊 ElevenLabs vocalizes the AI Professor's response via TTS
  • 👁️ A visibilitychange Integrity Tracker monitors for cheating (alt-tab detection)

Phase 5 — Grading & Report

Backend merges AI rubric scores with voice confidence metrics (WPM) and integrity penalties, then generates a professional PDF scorecard using fpdf2.

Tech Stack: Next.js · FastAPI · PostgreSQL + pgvector · Google Gemini · Vertex AI · Deepgram · ElevenLabs · LangChain · Clerk Auth

🧠 What I Learned

  • RAG architecture is everything. Without grounding the AI in the actual thesis text, every answer was a hallucination. The moment we added cosine similarity retrieval from pgvector, the grading accuracy skyrocketed.
  • Latency kills immersion. A 3-second delay between speaking and hearing the AI respond breaks the "real defense" illusion. We optimized the STT → RAG → LLM → TTS pipeline to keep round-trip latency under \( \sim 2 \) seconds.
  • Cheating is a real UX problem. When I first tested, every user instinctively alt-tabbed to check their notes. Adding the Integrity Tracker completely changed user behavior and made the simulation feel authentic.
  • Voice metrics reveal confidence. Words Per Minute (WPM) is a surprisingly strong proxy for how well a student actually understands their own research:

$$\text{WPM} = \frac{\text{Total Words Spoken}}{\text{Session Duration (min)}}$$

⚡ Challenges I Faced

Challenge How I Solved It
STT Latency Switched from REST-based transcription to Deepgram WebSockets for streaming, cutting latency by \( \sim 80\% \).
LLM Hallucinations Implemented strict RAG grounding — the AI can only reference text that exists in the uploaded PDF. Prompt engineering forces page-number citations.
PDF Parsing Edge Cases Academic PDFs are messy (multi-column layouts, LaTeX artifacts). I tuned PyMuPDF extraction with page-aware chunking to handle real-world thesis formats.
Browser Integrity Tracking The visibilitychange API behaves differently across Chrome, Firefox, and Safari. I normalized the event handling to ensure consistent cheating detection.
Audio Feedback Loops When the AI Professor speaks via TTS and the mic is still active, the system would "hear itself". I implemented automatic mic muting during TTS playback to prevent echo loops.

Built With

  • clerk-(authentication)
  • cosine-similarity-search
  • elevenlabs-(text-to-speech)
  • fastapi
  • fpdf2
  • google-gemini-2.0-flash
  • google-gemini-2.5-flash
  • google-vertex-ai
  • hnsw-vector-indexing
  • javascript
  • langchain
  • mediarecorder-api
  • next.js
  • pgvector-apis:-deepgram-(speech-to-text)
  • postgresql
  • pymupdf
  • python
  • rag
  • react
  • recharts
  • render
  • server-sent-events-(sse)
  • sqlalchemy
  • supabase
  • typescript
  • vercel
  • websockets
Share this project:

Updates