-
-
CyberTiger Dashboard
-
Track your Chatbot Conversation
-
Security Scan for LLM
-
Monitor your Conversation
-
Have a look into your Conversations Log
Inspiration
Nowadays businesses are putting AI chatbots on their websites, trained on their own documents: the help centre, the refund policy, sometimes a spreadsheet nobody meant to share however people tend to forget the risk of uploading sensitive data straight into the chatbot
We wanted the owner to have that view, in plain English.
What it does
Cyber Tiger watches every real conversation a chatbot has, at the retrieval layer: the question, the passages the bot read, and the answer. Each conversation runs through a set of detectors mapped to the OWASP Top 10 for LLM Applications.
- Data leaks. Canary tokens in confidential files, plus matching of values that exist only in confidential data (salaries, names, dates of birth, passport numbers), so a leak is caught even when the model rewrites it.
- Poisoned documents. Hidden instructions, invisible Unicode, HTML comments and tracking images, flagged the moment a passage is retrieved, whether or not the model obeys it.
- Uploads. Files a customer attaches in the chat, and documents an owner adds, are scanned before the bot can read them. Risky ones are quarantined.
- Plain-English rules. The owner writes rules such as "Only refer customers to [email protected]", and every answer is checked against them.
- Made-up answers and knowledge gaps. Answers that state things no document supports are flagged, and questions the documents don't cover ("Do you fly to Jakarta?") become insights for the owner.
- Trace-back. Every alert opens the full story: the question, the exact sentence that caused it, the answer, and a one-click quarantine.
As a proof of concept we made Skerry, a fictional Cork travel app as a use case. Its website chat connects to Cyber Tiger with an API token.
How we built it
- Backend: Python and FastAPI, with SQLite for traces, alerts, documents and tokens. Only token hashes are stored.
- Model: a real local LLM,
qwen2.5:1.5bthrough Ollama, so every answer and its timing is genuine. - Retrieval: TF-IDF over document chunks, which is deterministic, works offline at the venue, and keeps the demo reproducible.
- Detectors: plain, rule-based Python functions, each one testable on its own. For example, an answer counts as not backed by the documents unless the answer is a refusal.
Challenges we ran into
- "Timelimits" Our first bot was scripted, and a volunteer spotted that no real model answers that fast. We switched to a real local model to provide the context of the purpose.
- Real models don't copy canary tokens. For instance, asked for the CEO's salary, the model answered "€156,000 per year" which is not suitable for the chatbot purpose. We added matching of confidential values, and confidential-value checks to the scan.
- False positives. Our first knowledge-gap check flagged words like "earn" and "money" we cant fine tune it due to the time limit. In the future, tuning heuristics to be useful without being noisy took several rounds.
- Detection by name. We first marked a file confidential because "confidential" was in its name. A real export is called something like
bookings_export.csv, so now the upload scan classifies documents by what's inside them.
What we learned
- Detection is heuristic. Cyber Tiger gives visibility, not a guarantee, and we say so.
What's next
We would like to extend it into email and slack alerts, a weekly owner report, GDPR and EU AI Act evidence exports, other connectors, and a hosted EU version with per-chatbot monthly pricing.
