OptiLLMGrader

Generate and Grade exams using LLM

Comment

What if you ask AI to check an answer sheet? Will it be able to check it just like a Human would. Would it handle complex understanding like "definitions" should be exact and "examples" can be different than database? This is what LLM can do. We are familiar with power of Chat GPT. In this project we are using Lllama 2, an opensource LLM by facebook to correct papers with human level understanding.

Inspiration

In India, the ratio of student to teacher is not optimal. Usually we could see a single teacher teaching 100s of students. So I always wished to make a application that would address this issue. Now we have youtube to teach students but still assessment is something still done in traditional way.

  • A teacher will make a question paper and everyone would solve the same.
  • Some students are left behind as they need more time to understand specific topic but since time is a big issue and also manual effort required a teacher cannot focus on each and every student. The concept of personal attention is missing.
  • So to solve this issue I created this Web App

What it does

  • First it creates a question paper based on the previous attempts of the student.
  • If a student has done poorly in few questions they would be asked again in future assignments.
  • Then not only generation but also the correction of the paper is done

  • This is done by using OCR and LLM techniques. Powerful OCR is used to detect even the handwritten data and LLMs are used to check papers.

  • The best part is we can pass complex queries like for "defination" the answer should be exact, for "examples". if "example" mentioned in answer of particular topic is different from answer in database it would be fine

    • Thus we can create a paper generator and corrector with almost zero human effort and also get a human level accuracy.

How we built it

  • Llama 2 LLM
  • Python
  • Javascript
  • MongoDB
  • Taipy
  • OpenCV

Challenges we ran into

The challenge which I ran into was that it required almost an hour for LLM to generate prompt even on Google Colab.You can see in screenshot here So Later I had to switch to less parameters model

Share this project:

Updates