🎬 MakeAIdea - AI Learning Platform Based on VOD

MakeAIdea is an AI-powered learning platform that analyzes YouTube VOD content to generate summaries → create questions → compare AI and user answers → and construct a question expansion tree.
By leveraging LLMs, RAG (Retrieval-Augmented Generation), OCR, and Whisper, it provides an effective way to learn from video content.


🔧 Tech Stack

  • Frontend: Next.js (React-based)
  • Backend: FastAPI (Python)
  • Database: PostgreSQL
  • AI Engine: OpenAI GPT-4, Langchain
  • Vector Search: FAISS
  • Speech-to-Text: OpenAI Whisper
  • OCR: Tesseract

🧭 Architecture Overview

graph TD
  A[🎥 YouTube VOD Input] --> B[🧠 Whisper: Transcription + Timestamp Extraction]
  B --> C[📑 Summary Generation (LLM 1)]
  C --> D[❓ Question Generation (LLM 2)]
  D --> E[🤖 AI Answer Prediction (RAG)]
  E --> F[🧑 User Answer Input]
  F --> G[📊 AI Evaluation (LLM 3)]
  G --> H{Level < 3?}
  H -- Yes --> D
  H -- No --> I[✅ Finish]

📁 Directory Structure

MakeAIdea/
├── frontend/                # User Interface (Next.js)
├── backend/                 # Core functionality API (FastAPI)
├── docker/                  # Docker-related settings
├── .gitignore
├── README.md
└── ...

📌 Main Features

Feature Description
🎧 YouTube Audio Extraction Extract MP3 using yt-dlp
📝 Transcription & Summarization Use Whisper for transcription, GPT-4 for summarization
❓ Question Generation Generate questions using GPT-4 and prior context
💬 AI Answer Prediction Generate context-based answers using FAISS + RAG
🧑 User Input Users answer via frontend
📊 Answer Evaluation Compare AI and user answers with GPT-4
⏱️ Segment Tracking Locate relevant video segment via timestamp

🚀 Setup & Execution

1. Environment Variables

Create a .env file and set the following:

OPENAI_API_KEY=your_openai_api_key
DATABASE_URL=postgresql://username:password@localhost:5432/your_database

2. Install Dependencies

# Backend
cd backend
pip install -r requirements.txt

# Frontend
cd ../frontend
npm install

3. Initialize the Database

cd ../backend
alembic upgrade head

4. Run the Server

cd ..
docker-compose up --build

🧠 Role of LLMs

LLM Model Usage Function
LLM 1 From full transcription → summarization Summary via GPT-4
LLM 2 From summary → question / answer Generate questions & context-based answers
LLM 3 Compare AI and user answers GPT-4 based evaluation and scoring

🧑‍💻 Team

You can find the full project here:
👉 Seoul-AI-Hackathon/MakeAIdea

Built With

Share this project:

Updates