Assessment Integrity OS

An AI-powered assessment integrity platform that detects plagiarism, analyzes AI-generated content, and enables real-time exam monitoring through intelligent workflows and evidence-based insights.

Comment

Live Demo: link

Inspiration

As online examinations, digital assignments, and AI-powered writing tools become increasingly common, maintaining academic integrity has become a major challenge for educational institutions. Traditional assessment systems often struggle to detect plagiarism, identify suspicious examination behavior, and analyze AI-generated content efficiently. Manual verification is time-consuming, and relying on a single detection method can result in false positives or missed violations.

This inspired us to build Assessment Integrity OS (IntegrityOS), an AI-powered platform designed to bring plagiarism analysis, AI-assisted content detection, identity verification, and examination monitoring into one unified system. Our vision is to make digital assessments more secure, transparent, and fair while supporting evidence-based human decision-making.

What it does`

Assessment Integrity OS is a centralized platform that helps educational institutions manage and monitor assessment integrity through AI-assisted analysis and real-time monitoring.

Key features include:

  • Plagiarism Analysis: Identifies potentially similar content using text comparison, semantic similarity, and embedding-based analysis.
  • AI-Assisted Content Detection: Analyzes submitted text for patterns that may indicate AI-generated content, providing supporting insights rather than treating detection scores as definitive proof.
  • AI-Powered Proctoring: Organizes visual, behavioral, and environmental signals to identify events that may require review.
  • Identity Verification: Supports face-based identity checks during the assessment process.
  • Real-Time Monitoring: Uses real-time communication to deliver monitoring events and assessment updates.
  • Evidence-Based Review: Helps faculty review suspicious events, similarity findings, and supporting information before making decisions.
  • Student Portal: Provides access to assessments, submissions, reports, notifications, and communication.
  • Faculty Portal: Supports assessment management, student monitoring, integrity review, reporting, and analytics.
  • Admin Portal: Provides centralized management of users, assessments, platform analytics, and security operations.
  • Support Portal: Helps manage user assistance and communication.

The platform brings these capabilities together to create a more connected assessment workflow for students, faculty, administrators, and support teams.

How we built it

We designed IntegrityOS using a modular full-stack architecture that combines a modern web interface, backend APIs, database persistence, AI workflows, and real-time communication.

Frontend: React, TypeScript, Vite, Tailwind CSS, shadcn/ui, Radix UI, React Router, and Recharts for the user interface, dashboards, navigation, and analytics.

Backend: Node.js, Express.js, TypeScript, REST APIs, Zod validation, and Socket.IO for application services, request validation, and real-time updates.

Database: PostgreSQL and Prisma for structured data management and database access.

AI and analysis: LangChain and LangGraph-oriented workflows for coordinating analysis tasks, with LLM integrations such as Groq and Ollama, embedding-based semantic similarity, and computer-vision components where supported.

Core workflow:

  1. Students access an assessment through the student portal.
  2. The system manages assessment attempts and captures relevant submission or monitoring information.
  3. Submitted content can be processed for text extraction, chunking, and similarity analysis.
  4. Embeddings and semantic comparison help identify potentially matching passages.
  5. AI-assisted analysis provides additional context for reviewing submitted content.
  6. Proctoring workflows organize available identity, visual, behavioral, and environmental signals.
  7. Relevant events and analysis results are made available to authorized faculty for review.
  8. Reports and dashboards help faculty and administrators understand assessment activity and potential integrity concerns.

We structured the application around separate portals and reusable components so that each user role can access the functionality relevant to their responsibilities.

Challenges we ran into

Building an assessment integrity platform involves more than integrating AI models. We encountered several important engineering and design challenges:

  • Coordinating multiple AI workflows: Plagiarism analysis, content analysis, identity checks, and proctoring require different processing steps and outputs.
  • Real-time monitoring: Monitoring events and assessment updates need to be organized and delivered without disrupting the user experience.
  • False positives: Similarity scores, AI-content indicators, and suspicious behavior signals cannot independently prove misconduct. Designing for contextual review is essential.
  • Data consistency: Assessment attempts, submissions, monitoring events, and analysis results need to remain connected throughout the workflow.
  • Authentication and authorization: Different roles require different permissions, and sensitive endpoints must verify resource ownership.
  • Performance and scalability: Processing documents, embeddings, images, and real-time events requires careful resource management.
  • Privacy and security: Student submissions, identity information, and monitoring data require appropriate access controls, retention policies, and secure handling.
  • Integrating multiple technologies: Connecting the frontend, backend, database, AI components, and real-time communication into one coherent application required careful architectural planning.

These challenges highlighted the importance of combining intelligent analysis with secure engineering and responsible assessment practices.

Accomplishments that we're proud of

We are proud of developing a unified platform concept that brings several assessment integrity capabilities into a single application instead of treating them as disconnected tools.

Our key accomplishments include:

  • Designing a multi-portal experience for students, faculty, administrators, and support teams.
  • Building a modern dashboard-oriented interface with reusable UI components.
  • Structuring backend services and database models around assessments, attempts, users, and analysis workflows.
  • Integrating semantic similarity concepts into the plagiarism analysis pipeline.
  • Developing coordinated AI-oriented workflows for organizing proctoring signals and evidence.
  • Incorporating real-time communication capabilities for monitoring and assessment updates.
  • Creating a foundation for reporting, analytics, identity checks, and security monitoring.
  • Identifying opportunities to improve the platform through stronger authorization, privacy controls, and evidence-based review.

Most importantly, we established a foundation for treating assessment integrity as a complete workflow—from assessment creation and submission to analysis, review, and reporting.

What we learned

Developing IntegrityOS taught us that reliable AI systems require more than a model or a detection score. They need a complete system around them.

We learned the importance of:

  • Full-stack architecture: Designing clear boundaries between frontend components, backend APIs, database operations, and AI processing.
  • AI workflow orchestration: Breaking complex analysis into smaller, traceable stages that can be maintained and improved independently.
  • Semantic similarity: Understanding how embeddings can help identify related passages even when the wording differs.
  • Real-time systems: Managing live events and updates while maintaining a responsive user experience.
  • Security by design: Applying role-based permissions, ownership checks, input validation, file-upload restrictions, and secure configuration.
  • Responsible AI: Treating automated findings as indicators for investigation rather than definitive judgments about a student's intent.
  • Testing and evaluation: Recognizing the need for representative datasets, measured precision and recall, false-positive analysis, and end-to-end testing before making performance claims.

We also learned that transparency and human oversight are essential when AI-generated findings may affect students' academic outcomes.

What's next for Assessment Integrity OS

Our next goal is to strengthen the platform's reliability, security, explainability, and readiness for real-world institutional use.

Planned improvements include:

  • Evidence Intelligence Dashboard: A unified timeline of assessment events, similarity findings, supporting evidence, and review outcomes.
  • Integrity Case Management: Structured workflows for reviewing, clearing, flagging, escalating, and resolving potential integrity concerns.
  • Improved AI Evaluation: Benchmarking plagiarism analysis, AI-assisted content detection, and proctoring indicators against labeled datasets.
  • Stronger Access Controls: Enforcing assessment ownership, attempt ownership, role permissions, and eligibility checks across all relevant APIs.
  • Privacy Controls: Adding configurable data retention, secure evidence storage, consent flows, and clear access policies.
  • Enhanced Real-Time Monitoring: Improving event correlation and contextual analysis to reduce unnecessary alerts.
  • Human-in-the-Loop Review: Giving faculty transparent explanations and supporting evidence before any consequential decision is made.
  • Reliable Reporting and Analytics: Providing institution-level insights while protecting sensitive student information.
  • Scalability and Testing: Improving background processing, distributed rate limiting, automated tests, and production deployment practices.
  • Local AI Support: Exploring local model execution through tools such as Ollama where suitable, reducing dependence on external AI services for supported workflows.

Our long-term vision is to evolve Assessment Integrity OS into a secure, explainable, and scalable assessment integrity platform that helps institutions protect academic standards while respecting student privacy, fairness, and due process.

Problem & Inspiration

As online examinations, digital assignments, and AI-powered writing tools become increasingly common, maintaining academic integrity has become a growing challenge for educational institutions. Traditional assessment methods often struggle to identify plagiarism, AI-generated content, suspicious examination behavior, and identity-related issues efficiently. Manual review is time-consuming, and relying on a single detection signal can lead to false accusations or missed violations.

This inspired us to build IntegrityOS — an AI-powered Assessment Integrity Platform that brings plagiarism analysis, AI-assisted content detection, identity verification, and real-time proctoring into one unified system. Our goal is not simply to flag suspicious activity, but to provide contextual evidence, transparent reports, and actionable insights that help faculty make informed decisions.

We envisioned a platform where students can complete assessments in a secure environment, faculty can monitor examination integrity, and administrators can manage users, policies, analytics, and security operations. By combining AI workflows, semantic similarity analysis, real-time communication, and evidence-based review, IntegrityOS aims to make digital assessments more reliable, transparent, scalable, and fair for everyone.

Built With

Share this project:

Updates

Submission history