WhatsApp Viva

Oral understanding checks after homework on WhatsApp. GPT-5.6 asks personalized questions; students reply via voice notes; Whisper + GPT-5.6 generate teacher insights.

Comment

Inspiration

Traditional written homework is highly vulnerable to copying, sharing, or translation without true conceptual understanding. At the same time, teachers do not have the hours required to individually interview and verify the understanding of every student in their classes.

We built WhatsApp Viva to serve as a lightweight "oral defense" system that scales effortlessly. It meets students right where they already communicate—on WhatsApp—to test their understanding interactively and dynamically, giving teachers deep conceptual scorecards rather than just simple grades.

What it does

WhatsApp Viva acts as an automated, AI-powered teaching assistant:

  1. Submission: A student submits their written answers via a simple, mobile-friendly Next.js web form.
  2. Oral Defense Trigger: The system evaluates their answer and triggers a WhatsApp conversation via Twilio.
  3. Interactive Voice QA: OpenAI's GPT-5.6 generates 2–3 personalized, dynamic follow-up questions tailored specifically to that student's homework submission. The student replies directly on WhatsApp using voice notes.
  4. Whisper Transcription: The voice notes are stored in Cloudflare R2 and transcribed locally via Workers AI Whisper.
  5. Teacher Dashboard & Heatmap: GPT-5.6 scores the transcript against the homework rubric. The teacher gets an understanding heatmap, plain-language summaries of comprehension gaps, and full session transcripts.

How we built it

We utilized a robust and highly performant serverless stack:

  • Frontend: Next.js App Router deployed on Vercel for the student submission page and the detailed teacher dashboard.
  • Backend API: A fast Cloudflare Worker built with the Hono framework.
  • Database & Auth: Supabase (PostgreSQL) for schema, storage, and secure Teacher Login using Supabase Auth.
  • AI Engine: OpenAI GPT-5.6 (via the Responses API) for generating personalized questions and final viva evaluation.
  • Audio & Transcription: Cloudflare R2 for audio storage and Workers AI @cf/openai/whisper for fast local transcription.
  • Messaging: Twilio WhatsApp API for two-way messaging between the backend and students.

Challenges we faced

  • Twilio Webhook Verification: Securing the inbound webhook from Twilio to ensure authenticity on Cloudflare Workers required custom Cryptographic key validation.
  • State Machine Complexity: Structuring a robust state machine that successfully tracks when a student is in a viva, which question number they are answering, handling consent, and failing gracefully when they send text instead of voice notes.
  • Sandbox Constraints: Working within Twilio's WhatsApp Sandbox means users must join first with a code. We handled this by providing an "Open demo class" dashboard that works out-of-the-box using simulated fixtures.

What we learned

We discovered that Cloudflare Workers paired with Hono is incredibly lightweight and responsive for webhook integrations. Additionally, we found that combining cost-effective local AI models (like Workers AI Whisper) with state-of-the-art reasoning models (like GPT-5.6) yields a perfect balance of performance and analytical capability.

What's next for WhatsApp Viva

  • LMS Integration: Integrating directly with Canvas, Google Classroom, or Moodle to pull rubrics and push viva scores automatically.
  • Language Practice Mode: Expanding the voice agent to support foreign-language classrooms, verifying pronunciation, and offering real-time pronunciation tips.
  • Intelligent Intervention Alerts: Proactively alerting teachers when a student triggers a "red flag" or displays a severe conceptual gap in their audio response.

Built With

Share this project:

Updates