PromptRisk

Inspiration

Teams give AI agents real tools (email, a database, a shell, payments) and often rely on a paragraph of prompt text to keep them in check. A prompt reads like a rule, but it is advice, not a permission system. [YOU: what personally made you care about this, in your own words.]

What it does

PromptRisk is a local web app for developers and hackathon builders. You paste an agent prompt, and it lists what the prompt declares the agent can do. Each finding quotes your own words and links to a numbered statement, so you can check why a concern was raised.

It then runs one fixed, fictional, simulated attack against two setups side by side: what the prompt alone allows, and what a generated guardrail policy would allow. Every outcome is explained by a named rule, and nothing is actually executed.

The flow is: Review a prompt → evidence-linked findings → Improve and re-test → Build a prompt from five requirement fields → an editable draft that is reviewed independently before it counts as a result. A requirements check compares what you asked for with what the draft declares, using only validated findings.

The demo video uses bundled sample data. It is not live AI output.

How we built it

Next.js, React, TypeScript, Tailwind CSS, Zod and Vitest. A language model is used in only two places: as a reader that extracts capabilities with verbatim quotes, and as a drafter for Build mode. Both outputs are treated as untrusted, and every finding is checked against your original text. Classification, the attack simulation, the policy, the comparison and the requirements check are plain deterministic code.

Providers (Groq, OpenAI, Gemini) sit behind one adapter contract: one attempt per request, plus a single fallback to Groq for provider-side failures. The bundled sample and example need no key.

I planned first with the Devpost Learn Skill Pack: scope, PRD and spec, then ordered build slices, each checked with tests, a type check and a production build. [YOU: what you personally did and decided during planning and building.]

Challenges we ran into

• Honest wording. It is easy to build a tool that gives a prompt a clean bill of health. PromptRisk never does. Outcomes are simulated and labelled "prompt wording only, not enforced".
• Reader variance. The same model can read the same sentence differently on different runs, so findings and rule decisions are kept separate.
• Real providers. Rate limits, refusals and malformed output are handled as typed errors that can never look like a result.
• An intermittent failure. Reviewing a generated draft with a real Groq model failed twice, then passed once in a later smoke test. The root cause is unresolved. I added sanitized failure labels so a future failure can be classified, but that is a diagnostic, not a fix.

Accomplishments that we're proud of

• A four-action before/after comparison where every outcome is explained by a named, reproducible rule.
• A loop that separates what better wording changes from what only enforcement can change.
• 1,268 automated tests passing offline (mocked network) across 64 files, plus a clean-clone build. [YOU: anything else you're proud of.]

What we learned

Prompt wording and permission enforcement are different layers, and a tool that shows the difference is more useful than one that scores a prompt. Offline tests don't prove a live provider behaves the same way, so I recorded the failures instead of hiding them. [YOU: what you learned about planning first, building with an AI assistant, or anything else.]

What's next for PromptRisk

Find and fix the root cause of the intermittent draft-review failure. Test the OpenAI and Gemini adapters and the Groq fallback with real requests, since they have only been tested with a mocked network. Add more attack scenarios and capability kinds, and import a real tool registry so the Before column can reflect actual runtime configuration.

Built With

• next.js
• node.js
• react
• tailwind-css
• typescript
• vitest
• zod