Your AI chatbot is leaking,
and a free tool proves it
One Python file probes any chatbot endpoint for system prompt extraction, prompt injection, and jailbreaks, then writes an HTML report you can drop into a Jira ticket on Monday.
- Attack classes
- 15 mapped. 3 in the free tool.
- Runs on
- Any HTTP chatbot you own or are authorized to test.
- License
- MIT. One Python file, 855 lines.
- First finding
- About five minutes. No signup, no API key.
One probe: the bot swore it kept its rules, then printed an internal URL.
Every team shipped a chatbot, and nobody tests them
Burp Suite doesn't probe prompt injection. Nessus doesn't probe system prompt extraction. Garak covers both, but it's a research toolkit with a learning curve. This is the productized version: free and open source for the three most common attack classes, paid for the other twelve.
Three attack classes, each taking 5 to 15 prompts
All three are real attacks seen against production chatbots in 2025 and 2026. None require sophistication. Each has a real consequence when it lands.
System prompt extraction
Five prompt variants that get the bot to reveal its internal instructions. A successful pull leaks your prompt engineering, internal URLs, and the exact guardrails an attacker then works around.
Prompt injection
Five payloads that make the bot emit attacker-chosen output. Now it is an attack proxy: phishing your users in its own verified voice, emitting XSS downstream, dropping guardrails on command.
Jailbreak
Role-play frames (DAN, developer mode), hypothetical framing, and encoding bypasses (ROT13, leet) that evade the safety policy. The bot says what its policy forbids, with your brand in the screenshot.
One command, five minutes to first finding
No SaaS signup, no API key: a Python file and the URL of a chatbot you own, or have written authorization to test. Point it at the endpoint and read the report.
$ curl -O https://raw.githubusercontent.com/CVEasy/cveasy-chatbot-pentest/main/cveasy_chatbot_pentest.py $ python3 cveasy_chatbot_pentest.py \ --url https://your-bot.example.com/api/chat \ --output report.html CVEasy Chatbot Pentest. Probing https://your-bot.example.com/api/chat Rate limit: 1.0s · Max tokens: 200 · Timeout: 30.0s [1/3] System prompt extraction · 5 probes [2/3] Prompt injection · 5 probes [3/3] Jailbreak via role-play / encoding · 3 probes Summary: [TRIGGERED] System Prompt Extraction: 2 of 5 probes succeeded. [TRIGGERED] Prompt Injection: 3 of 5 probes succeeded. [clean] Jailbreak: No probe succeeded. Report written to report.html CVEasy AI covers 15 attack classes plus full TRIS scoring. https://cveasyai.com
The HTML report uses CVEasy editorial design. Drop it into a Jira ticket, a Slack thread, or an audit binder.
The free tool covers three of fifteen, and the other twelve are the moat
Running a chatbot in production, the free tool catches the bottom of the iceberg. The paid CVEasy AI app catches everything above the waterline, then scores it with the full 12-layer TRIS model.
| Capability | Free CLI | CVEasy AI |
|---|---|---|
| System prompt extraction | Yes · 5 probes | Yes · 50+ probes, multilingual |
| Prompt injection (direct) | Yes · 5 probes | Yes · 50+ probes |
| Jailbreak via role-play | Yes · 3 probes | Yes · 30+ probes |
| Indirect prompt injection (URLs, files, RAG) | No | Yes |
| Tool / function abuse | No | Yes |
| PII / conversation leakage | No | Yes |
| Cross-tenant leakage | No | Yes |
| Cost / resource exhaustion (recon only) | No | Yes |
| RAG poisoning | No | Yes |
| Hallucination weaponization | No | Yes |
| Multi-turn coercion (chain attacks) | No | Yes |
| TRIS scoring (12 layers) | No · single tier | Yes · full 12 layers |
| Continuous monitoring | No · one-shot | Yes · daily feed |
| Customer-deliverable HTML reports | Yes · basic | Yes · audit binder grade |
| Auto-discovery of chatbots across your surface | No | Yes |
Get told when the other twelve ship.
A new attack class lands every few weeks. Subscribe and we send a sample report and the upgrade pricing when the next class drops. Three to five emails a year.
No spam. Unsubscribe anytime.
Built by the team behind TRIS and BASzy.
The free probes are one slice of the same methodology that scores and validates exposures inside the paid CVEasy AI platform. Every line is readable, and the IP behind it is filed.
Built with Anthropic Claude
In the paid product, payload generation, AI Red-Team Mode, and customer brief authoring run on Anthropic Claude, and CVEasy is a Claude Partner. The app supports a bring-your-own Anthropic API key for predictable enterprise billing.
Patent-protected scoring and attack methodology
Ten patents filed on TRIS 12-layer scoring, breach and attack simulation, and AI chatbot pentest methodology. This free tool is one slice of that IP, released for the community.
Open source, so you can read every line
855 lines of Python in one file. Fork it, modify it, redistribute it. When a customer asks where the probes came from, you show them the source. No black box.
Run the free tool, and if it trips, we should talk
If your chatbot trips even one of the three free attack classes, see the other twelve before someone else does.