ValidForge

Inspiration

We were inspired by the growing gap between the promise of AI agents and their real-world reliability in enterprise settings. Companies are rapidly feeding internal knowledge (Slack threads, Notion pages, meeting notes) into agentic systems, but most tools stop at simple retrieval or structured extraction. The result is often “confident-looking garbage” — skills that look clean but contain hallucinations, missing evidence, or contradictions. We wanted to solve the root issue: how do you turn messy documents into trustworthy, agent-consumable skill artifacts without blindly trusting the LLM?

What it does

ValidForge lets users upload or paste a single enterprise artifact (PDF, text, Slack thread, or Notion page). It extracts a structured Skill Capsule (name, description, inputs, mastery criteria, sources), automatically flags uncertain fields using structural signals (no clear textual anchor, conflicting mentions, or missing evidence), and forces human review on every flagged field before the capsule is locked. Once validated, it generates 5–8 quiz questions and exports a clean, agent-ready JSON file.

How we built it

We created a complete set of high-fidelity Figma mockups that demonstrate the full end-to-end experience. The designs show:

Clean upload/paste interface Live extraction preview into the Skill Capsule JSON schema Uncertainty flagging UI with clear reasons (no textual anchor, conflicting sources, missing evidence) Interactive validation screen that requires explicit review on flagged fields Quiz generation and one-click JSON export The mockups were built to feel production-ready and fully interactive for the hackathon demo.

Challenges we ran into

The biggest challenge was avoiding the unreliable LLM self-reported confidence scores. We addressed this by using structural, source-verifiable uncertainty flags instead. Scope management was critical — we deliberately removed OAuth connectors, multi-source synthesis, and versioning features to keep the concept focused and demo-safe. We also had to ensure the validation step felt meaningful rather than a rubber-stamp by designing the UI to require explicit edits on uncertain fields.

Accomplishments that we're proud of

We delivered a polished, complete Figma prototype that clearly communicates the full product flow and differentiator. We created a defensible uncertainty detection approach that doesn’t rely on poorly calibrated LLM confidence numbers, and we produced a tight, hackathon-ready concept that directly solves a real enterprise pain point while remaining credible under Q&A.

What we learned

We learned that raw LLM confidence scores are poorly calibrated and should not be trusted as accuracy metrics. Structural, source-verifiable signals are far more reliable and explainable. We also learned the value of aggressive scoping and strong visual design — Figma mockups allowed us to show a complete, compelling experience even when working under tight time constraints.

What's next for ValidForge

We plan to add support for multiple source documents with explicit contradiction handling, integrate deeper enterprise connectors (Notion, Slack, Confluence), and expose the validated skill capsules for direct consumption by platforms such as Freshworks Agent Studio. Long-term, we want to explore automated re-validation triggers when source documents are updated.

Built With

• figma