trace

The work behind the work deserves to be seen.

Comment

Inspiration

The rise of generative AI has fundamentally changed how writing is created and evaluated. As tools like ChatGPT become commonplace, schools and universities are increasingly questioning whether submitted work is genuinely a student's own.

Unfortunately, the solutions available today often focus on the wrong problem. Most AI detection systems analyze only the final document and attempt to guess whether it was AI-generated. These systems are well known for producing false positives, leaving honest students with little evidence to defend themselves beyond saying, "I wrote it." But writing has never been about the final draft.

Real writing is messy. It includes deleted paragraphs, moments of hesitation, rewritten sentences, abandoned ideas, and gradual refinement over time. Those signals disappear once a document is submitted, even though they are often the strongest evidence of genuine authorship.

We started asking a different question: What if, instead of trying to detect AI, we preserved the story behind the writing?

That question became trace.

Rather than judging the finished essay, trace captures the creative process itself. By recording how a document evolves, it creates a transparent timeline of authorship that students can voluntarily share when needed. The goal isn't to police AI use or replace educators' judgment. It's to give honest students a fair way to demonstrate the work they actually did.

What it does

trace functions like any other document editor and is designed to benefit both students by eliminating the stress associated with work being falsely flagged as AI-generated and teachers by supplying them with an objective method of determining a work’s legitimacy through the milestone system.

Milestone System:

Milestones are trace’s way of identifying document changes. Each milestone contains a snapshot of the document, describing how it has changed since its previous state. They also contain important metadata such as if a bulk paste occurred (pastes of at least 50 words in length), the typing velocity (measured in words per minute), the time at which the milestone was saved, and the total number of words in the document when the milestone was created.

Whether or not a milestone is created depends on if trace believes the change was meaningful enough, which is determined by factors such as bulk paste, the time elapsed between milestones, and more. Typically, milestones are created each time 12 words or more have been changed in the document, meaning large documents could contain 40+ milestones. Thus, to better assist teachers, we developed a system using gpt-5.4-nano to summarize the milestones, taking advantage of their content and metadata, to accurately generate a score to determine how certain it is that the document is legit, with a passing score (a score greater than or equal to 70) indicating that the document is likely legit.

trace allows students to:

  • Create and edit documents
  • Join instructor assignments with an access code
  • Keep track of their assignments and their due dates
  • Submit assignments
  • Look at a stripped-down receipt of their milestones containing information like active writing time, revision history, paste activity, word count, and writing pace

trace allows instructors to:

  • Create, edit, archive, and delete assignments
  • Generate and share access codes for assignments
  • View information on their assignment dashboard to see which students submitted their work on time, late, or not at all
  • Review final documents as well as their revision history
  • Unsubmit student assignments in case they need to make corrections
  • Access the full receipt view, which contains the AI summarization feature

How we built it + How we used Codex

trace was built using the popular React framework Next.js, given the importance of server-side components and server actions in speeding up our agentic workflow. This means that Codex can directly embed database calls into the React code itself, through server-side components, and allows us to call complex logic with server actions without needing to expose our own API endpoints.

The database ORM we chose was Drizzle due to its interoperability with NeonDB, a free, serverless postgresql database (with extra emphasis on it being free due to us being a group of brokies). For authentication and securing our app, we used Clerk as it allows us, especially with the Clerk Codex agent skills, to ensure our app has industry standard security and integrates with Google sign-in without much effort on our parts.

To develop a consistent, beautiful, and functional frontend user interface, we used ShadCN’s prebuilt UI components alongside skills for TailwindCSS, ShadCN, and Next.js, which made it easy to ask Codex to make this arduous task extremely straightforward as well as keeping our codebase up-to-date with the latest best practices for each of these libraries. For the document editor we used blocknotejs, due to its ability to natively export PDFs, but heavily customized it to fit the aesthetic of a Google Docs-like editor. jsPDF was used for exporting PDF receipts.

Codex’s seamless integration with skills and plugins allowed it to act as a certified expert in anything we threw at it. The best example of this is our legitimacy AI, which was arguably the most important part of our application, but no one on the team had experience with the GPT API or the various models it offers. Using the OpenAI Developers Codex plugin, all we had to do was provide it a simple prompt describing the purpose of the legitimacy AI, and Codex was able to fully integrate it into our application and even generate an API key and choose a model, gpt-5.4-nano, which aligned with our needs of being low-cost without compromising on reasoning.

Challenges we ran into

  • Milestone summarization AI being too lenient: On documents where we were legitimately writing, we would get high scores, which was great, but for documents that were blatantly AI generated and copy pasted into trace, we would also be met with a high score. We initially assumed that this must’ve been an issue pertaining to the prompt itself, but after hours of troubleshooting, we were not getting any more accurate. It was to the point where we were considering getting rid of the scoring functionality all-together. However, as a last ditch effort, we decided to loosen the prompt we gave Codex for fixing our AI summarization, and it determined the issue was we were passing too little information to the model. It recommended we include document additions and removals instead of just passing in the metadata, and this fix resulted in immediate improvements.
  • Pages not updating when a delete button or save button was pressed: Using the Vercel and Next.js skills, Codex was able to solve the issue by adding a export const revalidate = 0; to files that contained dynamic data and using a call to revalidatePath(); to ensure that Next.js was not caching the dynamic pages.

Accomplishments that we're proud of

  • Accurate and smart AI milestone summarization system that knows the difference a student bulk pasting from AI or other sources and claiming it as their own versus using said external source with proper citations
  • AI summarization guardrails to ensure the instructor knows when the AI is unsure and manual review is necessary
  • Feature rich document editor with support for embedding links, images, videos, highlighting and altering text styles, PDF exports, and more
  • PDF exporting function for milestones that allows teachers to keep an external log of a students work and share it with other instructors
  • Detailed landing page and abundant onboarding-related features to ensure anyone can use trace correctly and take advantage of its many features
  • Unified and consistent user interface that establishes trace’s branding--no page sticks out from the rest

What we learned

  • AI prompt engineering: We learned the importance of setting up strong guardrails and providing the model with tons of examples to eliminate imprecission in our prompt and ensure that the model is fully able to understand what the application expects. We also learned about the importance of golden datasets in ensuring the model stays consistent.
  • Importance of AGENTS.md and skills.md files to direct AI and improve its outputs
  • Identified how to optimize our project by comparing different GPT model rate limits, costs per 1M tokens, and ideal use cases
  • An optimal analysis flow functions as a layered pipeline: prompts dictate the model’s behavior, server actions enforce runtime paths, and analysis modules must keep rules consistent across milestone receipts.
  • Importance of creating user workflows (in this case, for students and instructors) to define clear logic boundaries and build reusable components and shared APIs, effectively reducing code redundancy and speeding up development time

What's next for Trace

  • Integration with Google Docs to give users the ability to use a more familiar platform for writing their docs
  • trace API to allow our app to connect with external gradebooks and assignment platforms like Schoology and Canvas
  • Google Calendar integration so students and teachers have an easier way of seeing what assignments are due and being notified when they are
  • Integrate additional “must-flag categories” regarding hate speech, threats and incitement of violence, harassment, doxxing, etc. to ensure a safe environment for students and instructors.
  • Incorporate additional contextual rules in the cases of text discussing historical and political matters. For instance, if harmful text appears as an analytical quote in a political paper to explain the harms of hate speech, it may not need to be flagged.

Built With

Share this project:

Updates