KlugMind_Logo

🧠 KlugMind

Turn scattered notes, voice recordings, photos, and syllabi into a clear study plan—powered by AI that can run directly on the student’s device.

🌟 Inspiration

Studying is not only about understanding course material; it is also about managing deadlines, organizing notes, and knowing what to study next. Many students collect information everywhere: handwritten notes, PDFs, syllabus screenshots, voice notes, class photos, and long blocks of text. Turning all of that into a realistic plan can take almost as much time as studying itself.

That challenge inspired KlugMind: an AI-powered study companion that helps students transform raw academic material into structured tasks, flashcards, and quiz content.

The name combines "Klug"—the German word for smart—with "Mind." The goal is simple: make studying more organized, less overwhelming, and more active.

💡 What It Does

KlugMind is a Flutter mobile application that lets students input study material in multiple ways:

Typing or pasting notes and syllabus text
Recording or dictating notes with speech-to-text
Uploading PDFs
Uploading photos of notes or documents
Capturing material with the device camera

The app then processes the material through OCR, PDF extraction, speech recognition, and an LLM pipeline. It can identify assignments, exams, quizzes, projects, deadlines, and important academic information.

Depending on the content, KlugMind produces two helpful outcomes:

A dated study plan: If deadlines are detected, the app creates prioritized study blocks in the Today’s Plan screen.
An active-recall study kit: If no reliable dates are found, the app creates a flashcard deck and quiz items from the material.

For example, a student can paste a syllabus containing an exam date, a quiz, and a project deadline. KlugMind extracts the events, estimates study duration, assigns priority based on urgency and grade weight, and turns them into practical study tasks.

🛠 How We Built It

KlugMind was built as a Flutter application using Dart, with a strong focus on combining mobile development with practical AI features.

Mobile Development

The application interface was built with Flutter and Material Design. It includes:

A course setup experience
Notes and material intake screens
A daily study planner
Flashcard deck and study-session screens
A profile page
Light and dark themes
A theme toggle available throughout the app

Multi-Modal Study Input

Students do not always have their study material in the same format, so KlugMind supports several input methods:

Typed text for notes and syllabi
Voice dictation using speech_to_text
PDF text extraction using syncfusion_flutter_pdf
Image and camera OCR using Google ML Kit Text Recognition
An editable text area so students can correct OCR or transcription errors before generating content

The voice feature appends new dictated text instead of replacing existing notes, making it easier to collect ideas and lecture notes over time.

AI and LLM Processing

The main intelligence of KlugMind comes from a structured LLM pipeline.

After receiving material, the app:

Sanitizes the text and applies input-length limits.
Sends the material to an LLM with a prompt requesting structured JSON.
Extracts assignments, due dates, flashcards, and quiz items.
Validates the returned JSON before using it in the application.
Runs one repair prompt if the model returns malformed JSON.
Uses a regex-based fallback to search for dates near words such as exam, quiz, assignment, project, midterm, and final.

This makes the app more reliable than simply showing the model’s raw response.

On-Device and Local AI

KlugMind supports two AI backends:

On-device LLM inference through flutter_gemma, allowing the model to run locally after it is downloaded.
Local Ollama development mode, allowing the app to communicate with an LLM running on the developer’s computer.

For local development, the app can use models such as qwen2.5:3b through Ollama. For the final mobile workflow, the goal is to make AI generation as private and offline-friendly as possible.

Study Planning Logic

KlugMind does not only extract dates—it turns them into actionable study tasks.

The planning system follows clear rules:

Exams and projects receive 90-minute study blocks.
Quizzes receive 45-minute study blocks.
Other tasks receive 60-minute study blocks.
Tasks due in two days or fewer are marked Critical.
Tasks due within five days are marked High.
Tasks due within 10 days are marked Medium.
Tasks due later are marked Low priority.
Assignments with a grade weight of 20% or more receive a priority boost.
Overdue work is shown on the current day so it does not disappear from the plan.

This creates a realistic daily workflow instead of just listing deadlines.

🧩 Challenges We Ran Into

One of the biggest challenges was working with AI, machine learning concepts, and LLMs for the first time. KlugMind was my first major attempt at combining mobile development with AI, so I had to learn how local models work, how to write effective prompts, how structured JSON extraction works, and how to handle unreliable model responses.

A major technical challenge was making sure the AI output could safely be used inside the app. Language models can return incomplete JSON, unexpected formatting, or unclear dates. To improve reliability, I added JSON validation, a one-time repair prompt, and a regex fallback that looks for dates around important academic keywords.

Another challenge was balancing the capabilities of an AI model with the limitations of a mobile device. A model running directly on a phone has memory, storage, performance, and context-window limits. Because of that, the current on-device mode uses shorter input limits and generates fewer flashcards and quiz questions than the local Ollama development mode.

Handling different input formats was also challenging. Typed notes, speech-to-text, PDFs, and OCR images can all contain mistakes or incomplete text. The solution was to make the extracted content editable before sending it to the AI model.

Finally, time was a major factor. Some features, especially full PDF analysis and richer profile functionality, were started but could not be completely finished within the available development time.

🏆 Accomplishments We’re Proud Of

I am proud that KlugMind is more than a simple notes app or static planner. It connects several technologies into one practical study workflow:

Mobile development with Flutter and Dart
Speech-to-text for fast note capture
OCR for extracting text from photos
PDF text extraction
On-device AI experimentation
Local Ollama integration for development
Structured JSON generation and validation
Priority-based task planning
Local storage for study tasks and flashcard decks
Flashcard generation for active recall

One accomplishment I am especially proud of is building a fallback system instead of fully trusting the LLM. If the model’s JSON cannot be parsed, KlugMind sends one repair prompt. If the model fails to identify dated events, the application also searches for likely dates using regex patterns around academic terms.

I am also proud of implementing the logic that turns extracted academic events into practical study blocks. The app considers due dates, task type, and grade weight rather than treating every task equally.

Most importantly, this project helped prove that mobile applications and AI can work together in a useful, student-focused way—even without relying entirely on cloud APIs.

📚 What I Learned

KlugMind was my first time seriously working with AI, machine learning concepts, and LLMs. Through this project, I learned much more than just how to call a model from an app.

I learned how to:

Combine Flutter mobile development with AI-powered workflows
Work with local LLMs through Ollama
Explore on-device model execution using flutter_gemma
Design prompts that request structured JSON instead of plain text
Validate, parse, and repair LLM responses
Build fallback logic when AI output is unreliable
Process different forms of user input, including text, voice, PDFs, and images
Use OCR and speech recognition in a Flutter application
Build a deterministic planning system around AI-generated data
Think about mobile constraints such as model size, memory use, storage, performance, and offline access

The biggest lesson was that AI should not replace application logic. The model is useful for understanding unstructured study material, but the app still needs validation, rules, error handling, and a clear user experience to turn that output into something reliable.

🔮 What’s Next for KlugMind

KlugMind already supports local study-task storage and can run an AI model on-device after its initial download. Therefore, completely offline study planning is not just an idea—it is part of the direction already built into the app.

The next stage is to make that experience more complete, intelligent, and polished.

Better UI and User Experience

Improve the visual design and overall navigation flow.
Make the onboarding process smoother for first-time users.
Improve loading states while the AI model is processing study material.
Add clearer feedback for OCR confidence, model errors, and successful generation.
Refine the flashcard and Focus Mode experience.

Complete PDF Analysis

PDF text extraction is already part of the app’s pipeline, but full PDF analysis needs more work and testing. Future improvements will focus on:

Handling larger PDFs more reliably.
Improving extraction quality from complex layouts.
Processing longer academic documents in smaller chunks.
Allowing students to review extracted PDF text before generating study content.
Better identifying headings, deadlines, course topics, and assessment sections.

Interactive User Profiles

The current profile page includes placeholder statistics, course information, and reminder controls. The next version will make it fully interactive by adding:

Account creation and authentication
Personal profile information
Secure sign-in and sign-out
Saved progress across devices
Persistent courses and settings
Study statistics and performance history
Personalized reminder preferences

Study Streaks and Progress Tracking

A future version will track the number of days a student studies consistently. This can include:

Daily study streaks
Total study days
Completed tasks per week
Flashcards reviewed over time
Visual progress summaries
Motivation badges or milestones The goal is to help students see their consistency, not only their unfinished work.

Spaced Repetition for Flashcards

KlugMind currently supports flashcard ratings such as Again, Good, and Easy. The next step is to connect those ratings to a real spaced-repetition system. Future flashcard improvements include:

Scheduling cards based on recall performance
Showing difficult cards more often
Increasing intervals for well-known cards
Tracking review history
Building personalized review sessions
Using an algorithm such as SM-2 to calculate future review dates This will turn flashcards from a one-time activity into a long-term active-recall learning system.

Quiz Generation From Flashcards

KlugMind already generates quiz items and displays the quiz count. A future update will turn those generated items into a complete quiz experience.

Planned improvements include:

Multiple-choice quiz questions
True-or-false questions
Short-answer questions
Automatic answer checking where appropriate
Quiz scores and review summaries
Explanations for incorrect answers
Converting saved flashcards into quizzes automatically

Video Analysis and Summarization

Students often learn from recorded lectures, tutorials, and educational videos. A future version of KlugMind could accept video content or transcripts and generate:

Short lecture summaries
Key concepts
Important timestamps
Flashcards from the lecture
Quiz questions
Suggested study tasks based on the video content This would extend KlugMind beyond notes and documents into a broader AI learning assistant.

Automatic Course and Topic Detection

Another planned improvement is training or adapting the model to infer the course subject or topic from the material a student provides. For example, KlugMind could analyze:

Typed notes
A voice recording transcript
A captured image
Extracted PDF text

Then suggest a topic such as:

Mathematics
Physics
Programming
Biology
Cybersecurity
History

The student could confirm or edit the suggestion, and KlugMind could automatically add the material to the appropriate course list.

Larger Context and Long-Text Support

The current on-device model has intentional limits because of mobile storage, memory, and token constraints. Future work will focus on processing longer notes and documents more effectively. Potential improvements include:

Chunking long documents into smaller sections
Summarizing sections before final generation
Combining results from multiple chunks
Improving prompt design for long-context material
Supporting larger local models when hardware allows
Adding a configurable cloud or local-server option for more powerful processing

Model Storage and Memory Optimization

Running a language model on a phone creates an important storage challenge: model files can be large, while students may have limited device storage. Future optimization work will explore:

Smaller and more efficient model formats
Quantized models
Optional model downloads
Model deletion and re-download controls
Storage usage indicators
Better memory management during inference
Reducing the application footprint without reducing study quality

The long-term goal is to provide useful AI features while keeping KlugMind lightweight enough for real student devices.

🚀 Why KlugMind Matters

KlugMind is designed around a simple idea: students should spend more time learning and less time organizing what to learn. Instead of forcing students to manually convert a syllabus into a schedule, rewrite notes into flashcards, and remember every deadline, KlugMind helps transform existing study material into a guided learning workflow. By combining AI, OCR, speech recognition, PDF extraction, mobile development, active recall, and local-first design, KlugMind aims to become a smarter study companion that works with the way students already collect information.