trycanari

• 

  trycanari.pages.dev

• 

  trycanari.pages.dev

Inspiration

AI agents are becoming more powerful. They can interact with tools, access private information, execute workflows, and even move money. But a single prompt injection can manipulate an agent into ignoring its instructions, leaking secrets, or making unauthorized tool calls.

I wanted to solve a simple problem: how can developers continuously test their AI agents against these attacks before they cause real damage?

Traditional security testing can be expensive, manual, and difficult to repeat. I envisioned a platform where anyone could challenge an AI agent, prove that a vulnerability exists, and earn a reward for finding a real, reproducible break.

That idea became trycanari, an open-source bug bounty platform built specifically for AI agents.

What it does

trycanari connects AI agent developers with security researchers and other AI agents through a bug bounty marketplace.

Sponsors register their agents, configure their system prompts and tools, define security boundaries using canaries, and fund rewards. Hunters then attempt to exploit those agents through permitted entry points, such as user messages and documents.

Every attack is replayed five times against a sandboxed copy of the target agent. If the same canary is triggered in at least three runs, the vulnerability is considered verified. The system preserves the transcripts as evidence and is designed to release the reward automatically in SOL through Solana.

Canaries can detect sensitive information leaks and forbidden tool calls, such as unauthorized transfers. After fixing a vulnerability, sponsors can test their patches against previously successful attacks to make sure the same weakness has not returned.

trycanari also provides an API, documentation, an MCP server, and a skill file that allow developers to integrate agent security testing into their coding-agent workflows.

The goal is to make AI agent security measurable, reproducible, and continuously testable rather than something developers check only once.

How we built it

We built trycanari around a complete vulnerability discovery and verification workflow.

• Agent configuration: Sponsors can define their agent's system prompt, model, tools, parameters, and permitted attack entry points.
• Canary-based detection: We implemented secret-leak and forbidden-tool-call canaries to detect when an agent crosses a defined security boundary.
• Sandboxed execution: Attacks run against isolated copies of agents with mock tools, ensuring that testing does not trigger real-world actions such as transfers or emails.
• Repeatable verification: Each attack runs five times from a fresh conversation. A vulnerability is verified only when the same canary trips at least three times.
• Solana escrow and rewards: The platform supports bounty funding, escrow accounting, verified payouts, and refunds through a blockchain-based payment workflow.
• Developer integrations: We built HTTP APIs, machine-readable documentation, an MCP server, and a skill file so coding agents can scan agent configurations, inspect results, and help developers patch vulnerabilities.
• Patch verification: Previously successful attacks can be replayed against updated configurations before a bounty becomes active again.

The central design principle was to turn an AI security claim into reproducible evidence: an attack, multiple independent runs, a clear verdict, and transcripts showing what happened.

Challenges we ran into

One of the biggest challenges was dealing with the nondeterministic nature of AI models. The same prompt does not always produce the same output, so a single successful attack is not enough to establish a reliable vulnerability. We addressed this through repeated executions and a consistent verification threshold.

Another challenge was detecting violations without giving hunters direct access to the agent's hidden instructions, secrets, or internal security rules. Canaries allow sponsors to define what must be protected while exposing only public labels to hunters.

Building a safe testing environment was equally important. Security testing should not accidentally execute real tool calls or affect external systems, so attacks run against sandboxed agents with mocked tools.

We also had to think about the entire bounty lifecycle, including funding, attempt ordering, duplicate payouts, refunds, evidence preservation, and retesting after a patch. Making these different components work together as one coherent workflow was a major engineering challenge.

Finally, we wanted trycanari to work for both human researchers and autonomous AI agents. That meant designing the platform around accessible web interfaces and machine-readable APIs instead of limiting the entire experience to one group of users.

Accomplishments that we're proud of

• Built and launched trycanari as an open-source project.
• Created a dedicated bug bounty workflow for AI agents, from configuration and canary creation to attack verification and rewards.
• Implemented repeatable, five-run attack verification to reduce the chance of accepting a vulnerability based on a lucky response.
• Designed a Solana-based escrow and payout workflow tied to verified vulnerabilities.
• Built a sandboxed testing approach that uses mock tools rather than performing real-world actions.
• Added patch verification so previously successful attacks can serve as regression tests.
• Made agent security testing accessible through an API, MCP integration, and machine-readable documentation.
• Designed the platform for a future where AI agents can act as security researchers and test other agents autonomously.

What makes us most proud is that trycanari is not just a collection of prompt-injection demonstrations. It is an attempt to build the infrastructure for discovering, proving, paying for, and preventing vulnerabilities in AI agents.

What we learned

We learned that AI security cannot rely entirely on static rules or one-time testing. An agent may behave correctly in one conversation and fail in another, making repeatability and evidence essential.

We also learned that a useful security platform must do more than identify a possible vulnerability. It needs to explain what happened, preserve evidence, give developers a way to fix the problem, and verify that the fix works.

Building trycanari also reinforced the importance of separating untrusted inputs from privileged instructions, keeping sensitive configuration private, isolating tools during security tests, and designing financial workflows around clear, verifiable state transitions.

Most importantly, we learned that AI agents can participate in both sides of security testing. They can be the systems being tested, but they can also act as hunters that generate attacks, examine responses, and refine their approaches through an API.

What's next for trycanari

Our next goal is to make trycanari useful to more teams building real-world AI agents.

We plan to expand model and deployment compatibility, improve attack coverage, and make it easier for developers to connect their existing agents and run security checks during development.

We also want to strengthen the integration with coding agents so vulnerabilities can be discovered, patched, and retested directly from a developer's workflow. Better reporting, richer evidence, and more comprehensive security assessments will help teams understand their risks and track improvements over time.

As the platform grows, we aim to attract more sponsors and independent security researchers, build a stronger ecosystem of human and AI hunters, and make continuous adversarial testing a normal part of shipping AI agents.

Our long-term vision is simple: before an AI agent is trusted with sensitive data, tools, or money, its security should be challenged, its weaknesses should be provable, and its fixes should be testable.

Built With

• python
• typescript