Studdy buddy

Inspiration Students already upload everything, PDFs, slides, notes, syllabi , then still can’t trust a chatbot with it. Generic AI will fill gaps with outside knowledge, invent page numbers, and pick a side when the lecture and the syllabus disagree.

We wanted the opposite: an assistant that only speaks from your course, shows where it looked, and stays quiet when the evidence isn’t there.

What it does Upload course materials, ask a question, get a cited answer.

Answers only from retrieved course chunks, never from the model’s general knowledge Cites the real document, page, and section, copied from retrieval, not invented by the LLM Labels the claim as fact, inference, mixed, or unknown When sources conflict (lecture vs syllabus), returns conflicting_evidence and cites both sides When nothing supports the question, returns insufficient_evidence instead of hallucinating Treats uploaded text as untrusted data, so a planted “ignore your instructions” in a PDF cannot jailbreak the agent The path we optimized for:

Upload → Ask → Retrieve → Answer → Citation How we built it BoxLang bx-ai (v3.4.0) + Gemini, with deterministic orchestration around the model:

Validate the request (requestId, courseId, question) Search course materials (search_course_materials, max 6 chunks) If retrieval is empty, skip the model entirely Fence evidence with aiFence() so retrieved text cannot act as instructions Ask the model for a single JSON StudyAnswer Rebuild the response from a field whitelist: drop fake evidence IDs, copy citation metadata from retrieval, cap confidence, schema-check The model is trusted to choose which evidence IDs to cite and to write student-facing prose. Everything else — filenames, pages, schema shape, error codes — is application code.

Offline tests run with a fake retriever and a mock provider (no API key). A live demo script hits Gemini on five scenarios: direct answer, two sources, conflict, no evidence, and prompt injection.

Challenges we ran into Gemini in bx-ai 3.4.0 has no tool calling, so the agent cannot “decide” when to search. We call retrieval ourselves, then pass fenced evidence in. That also made citation validation possible: we know the allowed ID set before the model runs.

We also had to work around provider quirks (retired default model, params not applied unless passed per call, OpenAI-style temperature rejected by Gemini). Those are documented in INTEGRATION.md so the next person doesn’t rediscover them at 3 a.m.

Accomplishments we’re proud of Fabricated citations never reach the UI — verified live Empty retrieval never calls the model Prompt injection inside a chunk is ignored Conflict between lecture and syllabus is reported, not papered over 68 offline assertions, plus five live Gemini scenarios, all schema-valid What we learned Grounding is not a prompt. It’s a pipeline: retrieve first, fence the data, whitelist the output, copy metadata from the source of truth. The LLM is a reasoning step inside that pipeline, not the product

Built With

• boxlang