Inspiration

Every semester, students ask the same questions: I was sick on exam day, can I defer? What is the deadline for special consideration? If I withdraw after the census date, do I get my fees back? The answers exist on the university's own pages, but they are long, spread across sites, and different for different campuses, so people ask friends, group chats or a general chatbot, and get a paraphrase. A paraphrase with the wrong deadline costs a semester. We wanted a desk you can ask in your own words, in English or Chinese, that answers with the university's own sentence, says which campus it applies to, and proves the sentence is real.

What it does

Cairn is a desk that answers from an organisation's own published rules and quotes them word for word. This instance holds Monash University's public pages for students: special consideration and extensions, deferred and supplementary assessments, eExams, withdrawing, intermission and fees, census dates and Disability Support Services, for the Australian, Malaysian and Indonesian campuses. It is unofficial and not affiliated with the university.

  • You pick your campus, then ask in English or Chinese. The answer gives the bottom line in one or two sentences, with the deadline marked, and names the page it rests on and the campus it applies to.
  • Every point rests on a quote from the page. The server checks each quote word for word against the passage it cites before the answer is shown. If no quote survives, the answer is withheld.
  • Under every answer is the route it took: the English search text, the passages retrieved, their relevance scores and the campus each applies to.
  • The library lists all 58 pages by topic with a campus filter, and keeps every question the desk could not answer.
  • The evaluation page shows how well retrieval finds the right page, for seven methods, over 45 student questions.
  • It works without an AI key. A second engine, built the way assistants worked before large language models, scores every sentence of the retrieved pages with a small local model and shows the best ones as the answer, each exactly as the page wrote it, with its page and campus beneath. It never paraphrases. Without a key it answers by itself; with one, the reader switches between "AI answer" and "Local engine" and can compare the two on the same question.
  • When the pages do not answer, the desk drafts the email the reader would send to the department: the question in English, the closest page, and the documented address when a page gives one. Copy it or open it in a mail app; nothing is sent.

How we built it

  • Corpus: 58 public pages on monash.edu and monash.edu.my, read in a browser (the sites refuse scripted requests, and the pages are the university's, so they are not redistributed; the repository holds the page list and the browser script). Each page is split at its headings into 747 passages (900 characters, 150 overlap), each tagged with its campus and topic.
  • Retrieval: a Chinese question is first rendered in English by Claude Haiku 5.5, because the pages are English. The search text and the passages are embedded locally with multilingual-e5-small and indexed in Orama, filtered to the pages for every campus plus the chosen one. The 20 nearest passages are re-scored by a local cross-encoder (bge-reranker-base); the top 6 are kept.
  • Generation: Claude Haiku 5.5 with structured output. The source_id field is an enum of the retrieved passage ids, so the model cannot cite a source it was not given. The prompt asks for the bottom line, the deadline, the next step and the campus, and forbids inventing dates, fees, forms or contacts.
  • Verification: the server normalises quotes, dashes, character width and whitespace, then looks for every quote in its passage. The pages shown come from verified quotes only.
  • Interface: React, Vite and Mantine, fully bilingual, with the Refract theme the team chose from twelve samples. The campus is a native radio group drawn as chips, chosen once and remembered. The water light on the cover is a small WebGL shader; it pauses off-screen and stops under reduced motion.
  • Checks: axe-core on every screen in both languages at desktop and phone widths (0 violations), and Hallmark's design lint.

Challenges we ran into

  • Our first corpus was WCAG 2.2. It worked, and nobody needed it: the official quick reference and any general model answer those questions already. We threw the corpus away, kept the mechanism, and pointed it at rules people actually look up every semester, where the exact sentence and the deadline matter.
  • Chinese questions did not find the right page: "I was sick on exam day, can I defer" retrieved the overview and the documents page but not the deferral page. Rendering the question in English before the search fixed it: Chinese questions went from 18 to 19 of 20 with the right page in the top five, and MRR from 0.69 to 0.86.
  • Three campuses share most rules but not all. Searching only the chosen campus lost the shared pages; searching everything mixed up Malaysian and Australian dates. The answer was a filter that always includes the shared pages and adds the chosen campus, plus a campus label on every source so a reader can see when a rule is specific.
  • The pages are the university's. We decided not to redistribute them: the repository has the page list and a script that reads the pages in your own browser; the desk shows short excerpts with a link to each page; the public demo carries the quotes but not whole passages.

Accomplishments that we're proud of

  • A retrieval evaluation with real numbers over 45 student questions: the method in use finds the right page in the top five for 44 of 45 (MRR 0.839).
  • The local engine, with no model at all, answers 41 of the 45 and puts its bottom line on a gold page about six times in ten; the five questions the pages do not cover all come back "not covered".
  • A live smoke test of seven questions with campus: four answered with every quote verified, one honestly "not in these pages", one asking for the missing detail, and one prompt-injection attempt (a false deadline) that the desk declined to confirm.
  • The whole thing runs on a laptop CPU with one API key, and switching to another organisation's rules is a folder of pages and a prompt.

What we learned

  • Pick the corpus by need, not by licence. A clean licence made WCAG easy and useless; public pages with a copyright notice were harder to handle and worth it.
  • Grounding has to be a mechanism, not a prompt. The enum on source_id and the server-side check did more for trust than any instruction to "cite sources".
  • Measure retrieval separately from generation, and look at the misses. Every wrong answer we found was a retrieval miss, and the fix (translation) came from reading the trace of one bad answer.

What's next

  • Real student questions as the evaluation set, and an evaluation of the written answers, not only retrieval.
  • Boost a campus's own pages when they exist, so a Malaysia student asking about loan laptops gets the Malaysia page first.
  • A maintainer view: which pages changed since they were read, and which questions and pages collect "this was wrong" flags.

Disclosure

  • Built during the hackathon window: the project started on 9 November 2026 and changed corpus on 10 November; every commit is dated.
  • Source text: public pages of Monash University for students, (c) Monash University, read on 9 November 2026, quoted in short excerpts with a link to each page and not redistributed. Cairn is not affiliated with, endorsed by or maintained by Monash University.
  • Open-source components, models and fonts are listed with versions and licences in THIRD_PARTY_NOTICES.md in the repository.
  • Prior work: the word-for-word quote-verification approach was first used in our Ctrl+V entry TraceDesk Lite; Cairn is a separate project with its own corpus, retrieval pipeline and interface.
  • No user testing has been done yet; the 45 evaluation questions were written by the team.

Built With

Share this project:

Updates

Submission history