Evidence-Gated Answers

Entrant: Adnan Shamim Dar (individual), medical student, Università Politecnica delle Marche, Italy. Track: Automate Your Workflow.

1. What does the Skill do?

It answers a question from the user's own NAS library only when at least two of their files carry the answer, cites file and page for every sentence, and otherwise refuses and says what it searched.

2. Target user and problem.

Medical students, researchers and clinicians-in-training keep thousands of papers, guidelines and lecture notes on their NAS. AI assistants answer questions about such libraries fluently even when nothing in them supports the answer. In medicine a confident unsupported answer is worse than no answer.

3. Proposed solution.

A gate in front of the answer. The Skill first collects evidence from the folders the user names, counts how many distinct files genuinely address the question, and only then lets the document expert answer, restricted to those files.

4. Core user flow.

The user asks a question and names a folder. The scope is checked, evidence is collected, and the gate decides. The user sees either SUPPORTED (a short answer with a citation after every sentence) or REFUSED (the reason, what was searched, and the nearest files to read). Optionally, the cited answer is saved as a note after the user approves.

5. Difference from existing tools.

Retrieval tools make grounding likely but still answer when retrieval fails. This Skill makes refusal a first-class, successful outcome: no supporting files means no answer. Every sentence is traceable to a file the user owns, and uncited sentences are removed rather than shown.

6. NAS capabilities used.

Step 1, ug-filesystem (file-agent) checks the scope. Step 2, ug-knowledge collect-evidence (knowledge-agent) gathers sources, passages and coverage. Step 4, ug-document qa (document-agent) answers from the surviving files only. Step 5, ug-knowledge save-text (knowledge-agent) saves the note, with approval.

7. Technical approach.

A pure orchestration Skill (SKILL.md plus one reference) delegating only to registered experts, never calling the CLI directly. The Skill applies the gate (step 3) to returned results: at least two distinct files whose passage addresses the question itself; users may raise, never lower, that threshold. Read-only except the approved save; NAS local model preferred; no external services. The gate comes from my earlier Refusal Gate project, which applies the same rule to PubMed abstracts and scored 10/10 on a ten-question known-answer set.

8. Development status.

Implemented: the complete SKILL.md workflow, all five delegation states, empty and partial results, and the gate rules. It was checked against the Starter Kit's checker rules. Pending NAS validation: real expert outputs, citation format from document-agent, and timing.

9. Validation plan.

Core 1: a folder containing three guidelines that agree, asked a question they cover, should return SUPPORTED with all three cited. Core 2: a question none of the files addresses (for example, whether celery juice cures cancer) should return REFUSED with the searched scope. Edge case: one supporting file plus one unreadable PDF should return REFUSED and name the unreadable file.

10. Known limitations.

Whether a passage addresses the question is judged by a language model, so this is the weakest step. Its quality depends on collect-evidence coverage, scanned PDFs without text may be missed, and two agreeing files can still both be outdated.

Third-party components: none. The Refusal Gate concept is the entrant's own prior work; the Skill itself was written during the Hackathon.

Built With

• agents
• claude-code
• markdown
• ugos