Grader AI

Initial login screen - UB interface with Auth0 backend
Duo integrated MFA with Auth0 secure router
Upload all student's submissions, and grader's rubrics, and other source material for context
Complete AI grading scores adhering to rubrics for all students file by file, with reasoning, and a semantic plagiarism checker score

Inspiration

Grading often consumes hours that educators would rather spend teaching and supporting students, so we wanted to streamline the process. Grader AI automates initial rubric-based evaluation while keeping instructors in control with a human-in-the-loop final review.

What it does

Grader AI logs into the required domain using secure authentication flows, including Duo integration with Auth0, to ensure only authorized users access the system. Once authenticated, the user can upload student assignments along with rubric files or contextual guidelines for grading. The application supports multi-modal file uploads, including PDFs, code, notebooks, and video files. Depending on the type of student submission, the AI Agents take over to extract structured data from unstructured formats (PDF - section - wise data, code & notebooks - code chunks, video - accurate text transcripts). After robust chunking and batching techniques, the system passes the rubric and the assignment content to an OpenAI LLM (4.1 - mini) that acts as a judge, after the data is structured by a Gemini LLM (2.5 - flash). It then produces a detailed, rubric-aligned set of scores that the grader can review, approve or manually check. The system also supports robust semantic plagiarism checking using vector embeddings and similarity search which helps capture underlying patterns between students' assignments. The final output supports consistency, reduces grading time, and assists educators without replacing their judgment.

How we built it

LangChain to manage multi-step LLM workflows with Gemini and contextual reasoning.
OpenRouter for flexible access to a range of powerful LLM models.
React for an intuitive, responsive front-end interface.
Flask as the backend server handling API routes, file uploads, and model interaction.
Auth0 for multi-authentication management with Duo Mobile.
ElevenLabs for extracting transcripts from video files.

Challenges we ran into

Handling authentication across multiple identity providers and ensuring smooth Duo integration.
Managing secure file handling for assignments and rubrics, including size limits and formatting inconsistencies.
Iterative design to ensure the LLM consistently adheres to rubrics.
Avoiding hallucinations and ensuring the LLM sticks to evidence-based grading.
Coordinating front-end and back-end interactions for real-time feedback and quick uploads.
Ensuring reliability when dealing with diverse assignment formats (PDFs, text, images, etc.)

Accomplishments that we're proud of

Successfully integrating a fully authenticated login flow with Duo and Auth0.
Building a functional end-to-end grading pipeline that accepts real assignments.
Designing a system that keeps educators in control with human-in-the-loop grading.
Achieving consistent rubric-based grading from multiple LLM models using LangChain.
Implementing impactful Agentic AI architecture.

What we learned

Authentication workflows can be surprisingly tricky when you mix services.
Rubric-based grading requires careful prompt structuring and validation loops.
Human-in-the-loop design boosts reliability and trust in AI systems.
User experience matters—educators care about simplicity and clarity.
Iteration and testing on real assignments make a huge difference in tuning performance.
Implementing large scale AI workflows.