Inspiration
AssessTrace — evidence by design. Redesign assessment so the learning process leaves evidence. The instructor decides; the tool traces. Canvas MDS is its first adapter.
The problem behind AssessTrace emerged from my work as Director of the Master's in Data Science at Universidad del Desarrollo (UDD), my participation in university and faculty AI committees, and the UDD AI Workshop.
Generative AI created an urgent assessment problem: a polished final product no longer provides enough evidence of how students reasoned, made decisions, responded to feedback, verified AI output, or contributed individually. Faculty need to move from evaluating only the product toward evaluating the learning process—but redesigning an entire course and implementing that design in an LMS is difficult and time-consuming.
This educational need predates Build Week. During the event, beginning on July 16, 2026, I turned that context into a working product: AssessTrace, its Evidence by Design methodology, and Canvas MDS as its first LMS adapter.
What it does
AssessTrace helps faculty redesign assessment for AI-rich education.
Using Codex with GPT-5.6, it:
- reconciles learning outcomes, assessment plans, rubrics, institutional AI guidance, faculty constraints, and pedagogical knowledge;
- diagnoses what the current assessment can and cannot establish about learning;
- distinguishes objectives from activities;
- identifies reasoning, decisions, iteration, feedback response, individual contribution, and responsible AI use that remain invisible;
- proposes multiple redesign alternatives with workload, risks, and trade-offs;
- stress-tests the selected design against scenarios such as AI-generated work without understanding or unequal team participation;
- stops whenever a material decision requires instructor authority;
- converts confirmed decisions into traceable, portable course artifacts.
Canvas MDS then compiles the approved design into a Canvas LMS blueprint. Its deterministic Python engine validates weights, mappings, feedback loops, evidence coverage, pending decisions, source traceability, and Canvas safety constraints.
Inspection and dry-runs make zero mutations. The protected creation path requires explicit confirmation of the exact Canvas course and creates only unpublished structures. AssessTrace does not publish, delete, grade, alter enrollments, or retrieve student submissions.
What makes it genuinely generative
AssessTrace is not a text splitter or a template-filling utility.
GPT-5.6 is used where interpretation and pedagogical judgment are required: understanding heterogeneous evidence, detecting validity gaps, proposing viable alternatives, explaining trade-offs, and testing whether an assessment could be passed without demonstrating the intended learning.
The model does not make the final pedagogical decision. The faculty member confirms, modifies, or rejects every material choice.
This produces a deliberate separation of responsibilities:
- GPT-5.6 proposes and reasons.
- The instructor decides.
- The deterministic engine validates.
- Canvas MDS traces and safely implements the approved structure.
In the sanitized reference case, AssessTrace redesigns a course that initially placed 95% of the grade on final team products. The approved design contains 40% final-product evidence, 55% process checkpoints, and 5% individual verification—while preserving the authentic project and Demo Day.
How I built it
AssessTrace is implemented as a portable Codex plugin with three reusable skills:
- Configure: establishes a secure local Canvas connection without exposing credentials in chat.
- Redesign: applies Evidence by Design, enforces faculty decision gates, and produces traceable redesign artifacts.
- Manage: audits, dry-runs, and optionally creates an explicitly approved unpublished Canvas structure.
The architecture combines:
- Codex with GPT-5.6 for reasoning and orchestration;
- Python for deterministic validation and Canvas operations;
- portable JSON evidence contracts;
- the Canvas LMS REST API;
- operating-system keyring storage for credentials;
- a credential-free judge case with synthetic, student-free data;
- 49 automated regression tests;
- a 24-check offline judge demonstration.
The Python engine makes no hidden model calls. Its purpose is to make safety and traceability rules predictable, inspectable, and testable.
Judges can run the complete offline verification without a Canvas account, OpenAI API key, or student data:
python -m unittest discover -s plugins/canvas-mds/scripts -p "test_*.py" -v
python judge_demo.py
Challenges I faced
The hardest challenge was not generating course content. It was translating pedagogical principles into contracts that remain useful for generative reasoning while also being deterministically verifiable.
A forward test exposed an all-to-all evidence matrix that looked complete but did not represent genuine alignment. I converted that failure into a negative regression test: the validator now reconstructs and rejects the exact cartesian mapping that previously passed.
Another challenge was preserving instructor authority. When a faculty member modified a proposed option, the system initially represented the provenance of the modification ambiguously. The final design now separates authority from provenance: an applied decision must be confirmed, while a separate field records whether it was selected or modified.
I also had to keep the Canvas integration deliberately narrow. The MVP refuses unsafe writes, requires stable identifiers for pending decisions, excludes student data by design, and contains no publication or deletion command.
Accomplishments
I am proud that the project demonstrates both generative value and operational restraint:
- a complete process-centered assessment methodology;
- explicit faculty decision hard stops;
- traceable objective → indicator → evidence → instrument → procedure alignment;
- adversarial assessment-validity simulation;
- deterministic rejection of unsafe or semantically invalid profiles;
- zero-mutation audits and dry-runs;
- protected, idempotent creation of unpublished Canvas structures;
- 49 passing automated tests and 24 credential-free judge checks;
- a portable plugin, example profile, documentation, and reproducible distribution package.
What I learned
The strongest role for generative AI in education is not formatting existing information. It is helping a faculty member reason through ambiguity: what an assessment really demonstrates, which learning processes remain invisible, and which redesign is educationally valid and operationally feasible.
I also learned that generative reasoning and deterministic controls should complement each other. The model is valuable where understanding and proposing are required; deterministic code is valuable where a system must validate, refuse, or protect an institutional platform.
Finally, traceability is not only a compliance feature. It makes the collaboration between AI and faculty understandable: sources remain sources, proposals remain proposals, instructor decisions remain decisions, and unresolved work remains visible.
What's next
The next step is a faculty-facing interface that preserves the same reasoning gates and safety invariants without requiring command-line operation. A realistic short-term deployment model is for an instructional designer or program lead to operate AssessTrace alongside faculty.
Future work also includes:
- pilots across different UDD courses and disciplines;
- macOS and Linux validation;
- additional rubric and quiz capabilities;
- support for other LMS adapters;
- longitudinal evaluation of whether process-centered designs improve assessment validity and student learning.
AssessTrace does not try to replace faculty judgment or become another assessment platform. Its goal is narrower: make learning evidence visible, keep the instructor in control, and safely trace the approved design into the systems institutions already use.
Log in or sign up for Devpost to join the conversation.