Cognify AI

AI-powered personalized tutoring that turns PDFs, slides, and videos into grounded answers, adaptive quizzes, and learning paths tailored to each student's progress.

Comment

About the Project

💡 Inspiration

Traditional digital learning often feels like drinking from a firehose. Students and self-directed learners are overwhelmed by hundreds of pages of PDFs, dense lecture slides, complex diagrams, and hours of recorded lectures. On top of that, conventional AI tutors can generate confident but inaccurate answers, sometimes citing nonexistent chapters or misrepresenting important concepts.

We asked ourselves: What if study materials could organize themselves into an interconnected, interactive knowledge universe?

This question inspired us to build an AI-Powered Adaptive Learning Platform that transforms static learning resources into an intelligent and personalized study environment. Our goal was to help learners understand not only what they need to study, but also how concepts connect, which prerequisites they need to master, and where every AI-generated explanation comes from.

By combining multimodal AI, retrieval-augmented generation (RAG), interactive knowledge graphs, adaptive assessments, and spaced repetition, we set out to make learning more structured, engaging, and trustworthy.

🚀 What It Does

Our platform transforms textbooks, lecture slides, diagrams, and video content into an interactive learning experience.

  • Multimodal Document Ingestion: Processes PDFs, PowerPoint presentations (.pptx), diagrams, and video keyframes to extract meaningful learning content.
  • Visual Understanding: Uses Google Gemini to interpret complex diagrams, flowcharts, charts, and architectural schematics, converting visual information into searchable semantic descriptions.
  • Interactive Knowledge Graphs: Automatically identifies topics, subtopics, and prerequisite relationships, helping learners visualize how concepts connect and what they should study first.
  • AI Tutor with Verifiable Citations: Generates answers grounded in uploaded learning materials, with citations linked to source excerpts and textbook pages.
  • Adaptive Assessments: Creates context-aware quizzes, evaluates student performance, and identifies knowledge gaps.
  • Spaced Repetition: Estimates memory retention and schedules revision sessions to reinforce concepts at appropriate intervals.
  • Learning Analytics: Helps learners track topic mastery, monitor progress, and focus on areas that need improvement.

🛠️ How We Built It

We designed a modular, full-stack architecture that combines document processing, semantic search, multimodal AI, and personalized learning algorithms.

1. Vector Embeddings and Semantic Search

We use the all-MiniLM-L6-v2 transformer model to convert text chunks and image descriptions into dense vector embeddings with 384 dimensions.

To retrieve relevant learning content, we calculate cosine similarity between a student's query and the indexed document chunks:

$$\text{Cosine Similarity}(q, c) = \frac{q \cdot c}{\Vert{}q\Vert{}_2 \times \Vert{}c\Vert{}_2}$$

We apply a similarity threshold of 0.25 to filter out low-relevance chunks before constructing the context for the AI tutor:

$$\text{Similarity}(q, c) \ge 0.25$$

This retrieval process helps the tutor focus on relevant course materials instead of relying solely on the language model's internal knowledge.

2. Spaced Repetition and Memory Retention

To support long-term learning, we implemented a revision engine based on an exponential forgetting curve:

$$R(t) = e^{-t/S}$$

Where:

  • $R(t)$ represents estimated memory retention.
  • $t$ represents the time elapsed since the previous study session.
  • $S$ represents the concept stability factor, which adapts according to assessment performance.

This model helps schedule revision sessions based on estimated retention and the learner's demonstrated understanding.

3. Technology Stack

Frontend

  • React 19 and Vite for a responsive, interactive interface.
  • Custom CSS design tokens for a consistent visual experience.
  • Lucide icons for interface elements.
  • Canvas Confetti for celebrating learning milestones.

Backend

  • Python and FastAPI for API development.
  • Pydantic for request and response validation.
  • Argon2 for password hashing and JWT-based authentication.

Data and Retrieval

  • PostgreSQL for users, courses, resource pages, and assessments.
  • Qdrant for vector storage and semantic similarity search.
  • Tenant-aware filtering using course_id and owner_id to isolate learning resources.

AI and Document Processing

  • PyMuPDF for PDF text, page, and image extraction.
  • python-pptx for inspecting PowerPoint presentations.
  • Google Gemini (gemini-2.5-flash) for multimodal interpretation of diagrams and visual content.
  • all-MiniLM-L6-v2 for generating semantic embeddings.

4. Citation Verification

Trust is central to our AI tutor. We attach identifiers to retrieved chunks and validate citations against the set of chunks actually retrieved for a response.

Here is a simplified example of our citation validation logic:

def verify_citations(
    llm_citations: list[str],
    valid_chunk_ids: set[str]
) -> list[str]:
    """Remove citations that do not match retrieved chunk IDs."""
    return [
        chunk_id
        for chunk_id in llm_citations
        if chunk_id in valid_chunk_ids
    ]

## 🧗 Challenges We Faced

### 1. Understanding Multimodal Learning Materials
PDFs frequently combine text, tables, diagrams, and flowcharts on the same page. Extracting plain text alone often loses important spatial and visual relationships.

- **Our solution:** We used PyMuPDF to extract page content and image blocks, then leveraged Gemini's multimodal capabilities to generate descriptive captions. These captions were indexed alongside textual content as synthetic context chunks, preserving additional information about the original visual material.

### 2. Preventing Citation Hallucinations
AI models can generate plausible-looking references that do not correspond to actual source material. This is particularly problematic in educational settings, where learners need to trust the information they receive.

- **Our solution:** We associated retrieved context with unique identifiers and validated generated citation IDs against the retrieved chunk set. Invalid references were removed before citations were returned to the client. This verification approach improves citation integrity, although it does not independently guarantee that every generated explanation is factually correct.

### 3. Resolving Knowledge Graph Cycles
Automatically extracting prerequisite relationships can create circular dependencies. For example, Concept A might be identified as a prerequisite for Concept B, while Concept B is also identified as a prerequisite for Concept A.

- **Our solution:** We introduced directed acyclic graph (DAG) validation and topological sorting to identify invalid prerequisite structures before rendering the interactive knowledge graph. This ensures that valid learning paths can be represented in a consistent prerequisite order.

### 4. Balancing Retrieval Precision and Coverage
A strict similarity threshold helps reduce irrelevant context, but it can also exclude useful information when a student's question uses different terminology from the original material.

- **Our solution:** We implemented semantic retrieval with relevance filtering to prioritize useful chunks. Further improvements will focus on tuning retrieval thresholds and evaluating answer quality across different subjects and question types.

## 🏆 Accomplishments That We're Proud Of

- **Source-Grounded AI Tutoring:** Built a tutoring workflow that validates citation identifiers and connects explanations to retrieved learning materials.
- **End-to-End Multimodal Processing:** Combined PDFs, PowerPoint presentations, diagrams, and video keyframes into a unified learning-content pipeline.
- **Interactive Knowledge Mapping:** Transformed extracted concepts and prerequisite relationships into visual knowledge graphs.
- **Personalized Revision:** Integrated a memory-retention model to support adaptive revision scheduling.
- **Full-Stack Integration:** Connected a React frontend, FastAPI backend, PostgreSQL database, Qdrant vector engine, and Gemini-powered multimodal processing.
- **Learner-Centered Experience:** Developed a responsive, dark-mode-first dashboard with interactive graphs, assessment feedback, and learning milestones.

## 📚 What We Learned

- **Precision Matters More Than Context Size:** Sending entire textbooks to an AI model is not always the best approach. Carefully chunked documents, semantic retrieval, and relevant context can help produce more focused answers while reducing unnecessary information.
- **Visual Content Contains Valuable Knowledge:** Diagrams and flowcharts often communicate relationships that plain text cannot capture. Converting these visuals into descriptive semantic representations makes them easier to search and integrate into a unified retrieval pipeline.
- **Trust Requires More Than a Good Prompt:** Prompt engineering alone cannot guarantee reliable citations. Citation identifiers must be validated, retrieved evidence must be relevant, and generated explanations should be evaluated against their sources.
- **Learning Should Be Adaptive:** Different learners struggle with different concepts and forget information at different rates. Combining assessment results with spaced repetition provides a foundation for more personalized revision strategies.
- **Architecture and Data Isolation Matter:** Separating relational data, vector retrieval, document processing, and API services makes the platform easier to maintain and extend. Tenant-aware filtering is also essential when users upload private learning materials.

## 🔮 What's Next?

We plan to extend the platform with features that make learning more collaborative, accessible, and personalized.

- **Real-Time Voice Tutor:** Introduce low-latency conversational audio for interactive tutoring, spoken explanations, and language practice.
- **Collaborative Study Rooms:** Allow learners to share annotated knowledge graphs, create collaborative flashcards, and challenge one another with adaptive quizzes.
- **Learning Management System Integration:** Build integrations with Canvas and Moodle to import syllabi, course documents, and lecture recordings.
- **Improved Knowledge Graphs:** Develop more sophisticated prerequisite inference and personalized learning-path recommendations.
- **Advanced Learning Analytics:** Track learning progress over time and improve revision scheduling using assessment history and retention patterns.
- **Stronger Evaluation and Reliability:** Introduce systematic tests for retrieval relevance, citation correctness, answer faithfulness, and knowledge graph quality.

## 🌟 Our Vision

We envision a future where learning is not about navigating endless documents or memorizing disconnected facts. Instead, every learner should have access to a personalized knowledge ecosystem that explains complex ideas, reveals connections between concepts, identifies knowledge gaps, and provides evidence for every important answer.

Our AI-Powered Adaptive Learning Platform is a step toward that future: **turning fragmented learning materials into an interconnected, verifiable, and adaptive learning experience.**

Built With

Share this project:

Updates

Submission history