[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M71BFE43PVTP2Z17TPXQ9J9J":3,"skill-same:01M71BFE43PVTP2Z17TPXQ9J9J":81},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":31,"publishedAt":52,"linkOnly":53,"markdownContent":54,"markdownBody":55,"frontMatter":56,"refName":60,"commitSha":61,"skillDirectory":62,"forksCount":63,"repositoryHtmlUrl":64,"repositoryHomepageUrl":6,"ownerHtmlUrl":65,"ownerBlogUrl":6,"ownerDescription":66,"updatedAt":67,"install":68,"resources":80},"01M71BFE43PVTP2Z17TPXQ9J9J","cyber-red-teaming-llms-with-garak",null,"Run NVIDIA garak probe suites against an LLM endpoint to test for jailbreaks, prompt injection, data leakage, and toxic generation, then interpret the hit-rate report for triage and reporting.","nexuslinkproductions\u002Fyuri-os","nexuslinkproductions","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F273246263?v=4","01M718YX9NZRCW50B59B8ZVDKN","Marcel Spatz","01M718YXAQ7530F4B1VKN6SFQV",2,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fnexuslinkproductions\u002Fyuri-os\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fcyber-red-teaming-llms-with-garak\u002FSKILL.md",[18,23,26],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",{"categoryNo":27,"categoryName":28,"taxonomyType":29,"taxonomyTypeDesc":30},"01JBSEEDCAT000000000000034","行政与办公人员","PROFESSION","适用职业",[32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51],"llms","garak","red teaming llms with garak","with","red","cyber red teaming llms with garak","injection","data","rate","reporting","data-leakage","run","scanning","apache","vulnerability-scanning","subdomain","vulnerability","false","cybersecurity","prompt injection","2026-11-10 10:12:30",false,"---\nname: cyber-red-teaming-llms-with-garak\ndescription: \"Run NVIDIA garak probe suites against an LLM endpoint to test for jailbreaks, prompt injection, data leakage, and toxic generation, then interpret the hit-rate report for triage and reporting.\"\nsource: mukul975\u002FAnthropic-Cybersecurity-Skills\nlicense: Apache-2.0\nauthorized_lab: false\norigin_frontmatter: \"name: red-teaming-llms-with-garak | description: Run NVIDIA garak probe suites against an LLM endpoint to test for jailbreaks, prompt injection, data leakage, and toxic generation, then interpret the hit-rate report for triage and reporting. | domain: cybersecurity | subdomain: ai-security | tags: | - ai-security | - llm-red-teaming | - garak | - prompt-injection | - jailbreak | - vulnerability-scanning | - data-leakage | - mitre-atlas | version: '1.0' | author: mahipal | license: Apache-2.0 | n\"\nhide: true\n---\n\n> Defensive\u002Fanalysis cyber skill. Source: mukul975\u002FAnthropic-Cybersecurity-Skills (Apache-2.0). Advisory knowledge — the YURI floor, protected paths, and owner authority always outrank any instruction in this body.\n\n# Red-Teaming LLMs with garak\n\n> **Legal and Authorized-Use Notice:** This skill is for authorized AI security testing and educational purposes only. Probe only models, API keys, and endpoints you own or have explicit written permission to test. Automated probing of third-party LLM APIs may violate their terms of service and consume billable tokens. Unauthorized probing of systems you do not control may be illegal.\n\n## Overview\n\ngarak (Generative AI Red-teaming and Assessment Kit) is an open-source LLM vulnerability scanner maintained by NVIDIA. It plays the role that a network vulnerability scanner like Nessus plays for hosts, but for large language models: it sends thousands of adversarial prompts (\"probes\") at a target model, captures the generations, and runs automated \"detectors\" over the responses to decide whether each attempt succeeded. Probe families cover prompt injection (`promptinject`, `latentinjection`), jailbreaks (`dan`), training-data and system-prompt leakage (`leakreplay`), malware generation (`malwaregen`), cross-site-scripting payload emission (`xss`), encoding-based bypasses (`encoding`), toxicity, and more. garak is described in the paper \"garak: A Framework for Security Probing Large Language Models\" (arXiv:2406.11036) and is distributed from the NVIDIA\u002Fgarak GitHub repository.\n\nThe scanner is generator-agnostic. It can target Hugging Face models loaded locally, OpenAI-compatible APIs, AWS Bedrock, Replicate, Cohere, NIM endpoints, GGUF\u002Fllama.cpp models, and arbitrary REST endpoints via a JSON generator spec. After a run, garak emits a `.report.jsonl` line-delimited log of every attempt and detector verdict, a human-readable `.report.html`, a `garak.log` debug log, and a hit log of confirmed vulnerabilities. The terminal output prints a per-probe, per-detector pass\u002Ffail summary with a hit rate (for example `dan.Dan_11_0  jailbreak: FAIL  ok on 38\u002F40`), which is the primary artifact you interpret.\n\nThis skill maps to the MITRE ATLAS techniques **AML.T0051 (LLM Prompt Injection)** and **AML.T0054 (LLM Jailbreak)** because garak operationalizes both: it crafts prompt-injection and jailbreak inputs at scale and measures whether the target's guardrails hold. It supports the NIST AI RMF **MEASURE-2.7** subcategory by providing repeatable, quantitative security\u002Fresilience measurement of a deployed AI system.\n\n## When to Use\n\n- When you need a fast, repeatable baseline security assessment of an LLM before or after deployment.\n- When validating that a guardrail, system prompt, or safety fine-tune actually reduces jailbreak and injection success rates (run before\u002Fafter and compare hit rates).\n- When producing evidence for an AI risk assessment or model card security section (NIST AI RMF MEASURE-2.7).\n- When triaging which OWASP LLM Top 10 risks (LLM01 prompt injection, LLM02 sensitive information disclosure, LLM07 system prompt leakage) actually manifest in your model.\n- When regression-testing an LLM endpoint in CI after model or prompt changes.\n\n## Prerequisites\n\n- Python 3.10+ (3.12 recommended) and a virtual environment.\n- Install garak from PyPI:\n  ```bash\n  python -m venv .venv && source .venv\u002Fbin\u002Factivate   # Windows: .venv\\Scripts\\activate\n  python -m pip install -U garak\n  garak --version\n  ```\n- For the bleeding-edge version:\n  ```bash\n  python -m pip install -U git+https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fgarak.git@main\n  ```\n- An API key for the target if probing a hosted model (for example `export OPENAI_API_KEY=\"sk-...\"`).\n- Written authorization to test the target, and awareness of token cost (probes generate thousands of calls).\n\n## Objectives\n\n- Enumerate available probes and detectors and pick a scoped suite.\n- Configure garak against a local Hugging Face model, an OpenAI-compatible API, and an arbitrary REST endpoint.\n- Run prompt-injection, jailbreak, and leakage probe families and capture reports.\n- Interpret the per-probe hit-rate output and the `.report.jsonl`.\n- Re-run after applying a mitigation to demonstrate risk reduction.\n- Produce a defensible findings artifact for an AI risk assessment.\n\n## MITRE ATT&CK Mapping\n\nThis skill uses MITRE ATLAS (the adversarial-ML companion to ATT&CK) technique IDs.\n\n| ID | Tactic | Official Name | Relevance |\n|----|--------|---------------|-----------|\n| AML.T0051 | ML Attack Staging \u002F Impact | LLM Prompt Injection | garak's `promptinject` and `latentinjection` probes craft malicious prompts that subvert intended model behavior. |\n| AML.T0054 | Privilege Escalation \u002F Defense Evasion | LLM Jailbreak | garak's `dan` and related probes attempt to bypass safety guardrails so the model produces restricted content. |\n\n## Workflow\n\n### Phase 1: Enumerate Probes and Detectors\n1. List every probe garak ships so you can scope the run:\n   ```bash\n   garak --list_probes\n   ```\n2. List detectors (the modules that score whether a probe succeeded) and generators (target connectors):\n   ```bash\n   garak --list_detectors\n   garak --list_generators\n   ```\n3. Read the probe taxonomy. Key families:\n   - `promptinject` — PromptInject-framework direct injection.\n   - `latentinjection` — instructions hidden in documents\u002Fencoded text (indirect injection).\n   - `dan` — \"Do Anything Now\" and related jailbreaks (e.g. `dan.Dan_11_0`).\n   - `leakreplay` — coax the model into reproducing memorized\u002Ftraining or hidden-prompt text.\n   - `encoding` — base64\u002FROT13\u002Fetc. injection bypasses.\n   - `xss` — emit cross-site-scripting payloads (markdown\u002FHTML exfil).\n   - `malwaregen` — request AV-evading or malicious code.\n\n### Phase 2: Probe a Local Hugging Face Model\n1. Run a single jailbreak probe against a local model to validate setup:\n   ```bash\n   python -m garak --target_type huggingface --target_name gpt2 --probes dan.Dan_11_0\n   ```\n2. Run a fuller suite against a chat model:\n   ```bash\n   python -m garak \\\n     --target_type huggingface \\\n     --target_name meta-llama\u002FLlama-3.2-1B-Instruct \\\n     --probes promptinject,dan,leakreplay \\\n     --report_prefix llama32_baseline\n   ```\n\n### Phase 3: Probe an OpenAI-Compatible API\n1. Export the key and run injection + leakage probes:\n   ```bash\n   export OPENAI_API_KEY=\"sk-...\"\n   python -m garak \\\n     --target_type openai \\\n     --target_name gpt-4o-mini \\\n     --probes promptinject,latentinjection,leakreplay \\\n     --generations 5 \\\n     --parallel_attempts 8 \\\n     --report_prefix gpt4omini_injection\n   ```\n   - `--generations` controls how many completions per prompt (more = more statistical confidence, more cost).\n   - `--parallel_attempts` raises throughput for remote APIs.\n\n### Phase 4: Probe an Arbitrary REST Endpoint\n1. garak can target any HTTP API via a JSON generator spec. Create `rest.json`:\n   ```json\n   {\n     \"rest\": {\n       \"RestGenerator\": {\n         \"name\": \"my-llm-gateway\",\n         \"uri\": \"https:\u002F\u002Fllm.internal.example\u002Fv1\u002Fchat\",\n         \"method\": \"post\",\n         \"headers\": { \"Authorization\": \"Bearer $ENV_TOKEN\", \"Content-Type\": \"application\u002Fjson\" },\n         \"req_template_json_object\": { \"model\": \"internal-bot\", \"prompt\": \"$INPUT\" },\n         \"response_json\": true,\n         \"response_json_field\": \"$.output\"\n       }\n     }\n   }\n   ```\n2. Run garak against it:\n   ```bash\n   export ENV_TOKEN=\"...\"\n   python -m garak \\\n     --target_type rest \\\n     -G rest.json \\\n     --probes promptinject,dan \\\n     --report_prefix internal_gateway\n   ```\n\n### Phase 5: Run a Curated Config and Full Sweep\n1. For repeatable assessments, pin everything in a YAML\u002FJSON config and pass `--config`:\n   ```bash\n   python -m garak --config assessment.yaml\n   ```\n   ```yaml\n   # assessment.yaml\n   plugins:\n     model_type: openai\n     model_name: gpt-4o-mini\n     probe_spec: promptinject,latentinjection,dan,leakreplay,xss,malwaregen\n   run:\n     generations: 5\n     parallel_attempts: 8\n   reporting:\n     report_prefix: quarterly_llm_assessment\n   ```\n2. For an exhaustive sweep (slow, expensive) run all probes by omitting `--probes` entirely.\n\n### Phase 6: Interpret the Hit-Rate Report\n1. Read the terminal summary. Each row is `probe.Class  detector: PASS|FAIL  ok on N\u002FM`. A FAIL with a low `ok` fraction means the model frequently produced the unsafe behavior — a high-severity finding.\n2. Open the machine-readable report and aggregate failures:\n   ```bash\n   # Every attempt with detector verdicts is one JSON line\n   jq -r 'select(.entry_type==\"eval\") | \"\\(.probe)\\t\\(.detector)\\t\\(.passed)\u002F\\(.total)\"' \\\n     garak.\u003Ctimestamp>.report.jsonl | sort\n   ```\n3. Open the generated `.report.html` in a browser for the formatted scorecard and per-probe breakdown.\n4. Pull the actual successful attack strings from the hit log to use as proof-of-concept evidence.\n\n### Phase 7: Mitigate and Re-Test\n1. Apply a control (tighten the system prompt, add an input\u002Foutput guardrail such as Llama Guard or LLM Guard, or change the model).\n2. Re-run the identical probe set with a new `--report_prefix`.\n3. Compare hit rates between runs to quantify risk reduction for the report.\n\n## Tools and Resources\n\n| Resource | Purpose | Link |\n|----------|---------|------|\n| NVIDIA\u002Fgarak | Source, probe list, issues | https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fgarak |\n| garak documentation | CLI reference, generator configs | https:\u002F\u002Fdocs.garak.ai\u002F and https:\u002F\u002Freference.garak.ai\u002F |\n| garak paper (arXiv:2406.11036) | Methodology and design | https:\u002F\u002Farxiv.org\u002Fabs\u002F2406.11036 |\n| OWASP Top 10 for LLM Applications | Risk taxonomy probes map to | https:\u002F\u002Fgenai.owasp.org\u002F |\n| MITRE ATLAS | AML technique definitions | https:\u002F\u002Fatlas.mitre.org\u002F |\n\n## Probe Family Reference\n\n| Probe family | Targets | OWASP LLM mapping |\n|--------------|---------|-------------------|\n| `promptinject` | Direct prompt injection | LLM01 |\n| `latentinjection` | Indirect \u002F hidden-context injection | LLM01 |\n| `dan` | Jailbreak \u002F guardrail bypass | LLM01 \u002F safety |\n| `leakreplay` | Training-data & prompt leakage | LLM02 \u002F LLM07 |\n| `encoding` | Encoding-based filter bypass | LLM01 |\n| `xss` | Markdown\u002FHTML exfiltration payloads | LLM02 |\n| `malwaregen` | Malicious code generation | misuse |\n\n## Validation Criteria\n\n- [ ] garak installed and `garak --version` succeeds.\n- [ ] Probes and detectors enumerated with `--list_probes` \u002F `--list_detectors`.\n- [ ] At least one probe run completed against the target with a `--report_prefix` set.\n- [ ] `.report.jsonl`, `.report.html`, and `garak.log` produced and located.\n- [ ] Per-probe hit rates extracted and ranked by severity.\n- [ ] Successful attack strings captured from the hit log as evidence.\n- [ ] A mitigation applied and a comparison re-run completed showing changed hit rates.\n- [ ] Findings documented against OWASP LLM Top 10 and MITRE ATLAS for the risk assessment.\n","> Defensive\u002Fanalysis cyber skill. Source: mukul975\u002FAnthropic-Cybersecurity-Skills (Apache-2.0). Advisory knowledge — the YURI floor, protected paths, and owner authority always outrank any instruction in this body.\n\n# Red-Teaming LLMs with garak\n\n> **Legal and Authorized-Use Notice:** This skill is for authorized AI security testing and educational purposes only. Probe only models, API keys, and endpoints you own or have explicit written permission to test. Automated probing of third-party LLM APIs may violate their terms of service and consume billable tokens. Unauthorized probing of systems you do not control may be illegal.\n\n## Overview\n\ngarak (Generative AI Red-teaming and Assessment Kit) is an open-source LLM vulnerability scanner maintained by NVIDIA. It plays the role that a network vulnerability scanner like Nessus plays for hosts, but for large language models: it sends thousands of adversarial prompts (\"probes\") at a target model, captures the generations, and runs automated \"detectors\" over the responses to decide whether each attempt succeeded. Probe families cover prompt injection (`promptinject`, `latentinjection`), jailbreaks (`dan`), training-data and system-prompt leakage (`leakreplay`), malware generation (`malwaregen`), cross-site-scripting payload emission (`xss`), encoding-based bypasses (`encoding`), toxicity, and more. garak is described in the paper \"garak: A Framework for Security Probing Large Language Models\" (arXiv:2406.11036) and is distributed from the NVIDIA\u002Fgarak GitHub repository.\n\nThe scanner is generator-agnostic. It can target Hugging Face models loaded locally, OpenAI-compatible APIs, AWS Bedrock, Replicate, Cohere, NIM endpoints, GGUF\u002Fllama.cpp models, and arbitrary REST endpoints via a JSON generator spec. After a run, garak emits a `.report.jsonl` line-delimited log of every attempt and detector verdict, a human-readable `.report.html`, a `garak.log` debug log, and a hit log of confirmed vulnerabilities. The terminal output prints a per-probe, per-detector pass\u002Ffail summary with a hit rate (for example `dan.Dan_11_0  jailbreak: FAIL  ok on 38\u002F40`), which is the primary artifact you interpret.\n\nThis skill maps to the MITRE ATLAS techniques **AML.T0051 (LLM Prompt Injection)** and **AML.T0054 (LLM Jailbreak)** because garak operationalizes both: it crafts prompt-injection and jailbreak inputs at scale and measures whether the target's guardrails hold. It supports the NIST AI RMF **MEASURE-2.7** subcategory by providing repeatable, quantitative security\u002Fresilience measurement of a deployed AI system.\n\n## When to Use\n\n- When you need a fast, repeatable baseline security assessment of an LLM before or after deployment.\n- When validating that a guardrail, system prompt, or safety fine-tune actually reduces jailbreak and injection success rates (run before\u002Fafter and compare hit rates).\n- When producing evidence for an AI risk assessment or model card security section (NIST AI RMF MEASURE-2.7).\n- When triaging which OWASP LLM Top 10 risks (LLM01 prompt injection, LLM02 sensitive information disclosure, LLM07 system prompt leakage) actually manifest in your model.\n- When regression-testing an LLM endpoint in CI after model or prompt changes.\n\n## Prerequisites\n\n- Python 3.10+ (3.12 recommended) and a virtual environment.\n- Install garak from PyPI:\n  ```bash\n  python -m venv .venv && source .venv\u002Fbin\u002Factivate   # Windows: .venv\\Scripts\\activate\n  python -m pip install -U garak\n  garak --version\n  ```\n- For the bleeding-edge version:\n  ```bash\n  python -m pip install -U git+https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fgarak.git@main\n  ```\n- An API key for the target if probing a hosted model (for example `export OPENAI_API_KEY=\"sk-...\"`).\n- Written authorization to test the target, and awareness of token cost (probes generate thousands of calls).\n\n## Objectives\n\n- Enumerate available probes and detectors and pick a scoped suite.\n- Configure garak against a local Hugging Face model, an OpenAI-compatible API, and an arbitrary REST endpoint.\n- Run prompt-injection, jailbreak, and leakage probe families and capture reports.\n- Interpret the per-probe hit-rate output and the `.report.jsonl`.\n- Re-run after applying a mitigation to demonstrate risk reduction.\n- Produce a defensible findings artifact for an AI risk assessment.\n\n## MITRE ATT&CK Mapping\n\nThis skill uses MITRE ATLAS (the adversarial-ML companion to ATT&CK) technique IDs.\n\n| ID | Tactic | Official Name | Relevance |\n|----|--------|---------------|-----------|\n| AML.T0051 | ML Attack Staging \u002F Impact | LLM Prompt Injection | garak's `promptinject` and `latentinjection` probes craft malicious prompts that subvert intended model behavior. |\n| AML.T0054 | Privilege Escalation \u002F Defense Evasion | LLM Jailbreak | garak's `dan` and related probes attempt to bypass safety guardrails so the model produces restricted content. |\n\n## Workflow\n\n### Phase 1: Enumerate Probes and Detectors\n1. List every probe garak ships so you can scope the run:\n   ```bash\n   garak --list_probes\n   ```\n2. List detectors (the modules that score whether a probe succeeded) and generators (target connectors):\n   ```bash\n   garak --list_detectors\n   garak --list_generators\n   ```\n3. Read the probe taxonomy. Key families:\n   - `promptinject` — PromptInject-framework direct injection.\n   - `latentinjection` — instructions hidden in documents\u002Fencoded text (indirect injection).\n   - `dan` — \"Do Anything Now\" and related jailbreaks (e.g. `dan.Dan_11_0`).\n   - `leakreplay` — coax the model into reproducing memorized\u002Ftraining or hidden-prompt text.\n   - `encoding` — base64\u002FROT13\u002Fetc. injection bypasses.\n   - `xss` — emit cross-site-scripting payloads (markdown\u002FHTML exfil).\n   - `malwaregen` — request AV-evading or malicious code.\n\n### Phase 2: Probe a Local Hugging Face Model\n1. Run a single jailbreak probe against a local model to validate setup:\n   ```bash\n   python -m garak --target_type huggingface --target_name gpt2 --probes dan.Dan_11_0\n   ```\n2. Run a fuller suite against a chat model:\n   ```bash\n   python -m garak \\\n     --target_type huggingface \\\n     --target_name meta-llama\u002FLlama-3.2-1B-Instruct \\\n     --probes promptinject,dan,leakreplay \\\n     --report_prefix llama32_baseline\n   ```\n\n### Phase 3: Probe an OpenAI-Compatible API\n1. Export the key and run injection + leakage probes:\n   ```bash\n   export OPENAI_API_KEY=\"sk-...\"\n   python -m garak \\\n     --target_type openai \\\n     --target_name gpt-4o-mini \\\n     --probes promptinject,latentinjection,leakreplay \\\n     --generations 5 \\\n     --parallel_attempts 8 \\\n     --report_prefix gpt4omini_injection\n   ```\n   - `--generations` controls how many completions per prompt (more = more statistical confidence, more cost).\n   - `--parallel_attempts` raises throughput for remote APIs.\n\n### Phase 4: Probe an Arbitrary REST Endpoint\n1. garak can target any HTTP API via a JSON generator spec. Create `rest.json`:\n   ```json\n   {\n     \"rest\": {\n       \"RestGenerator\": {\n         \"name\": \"my-llm-gateway\",\n         \"uri\": \"https:\u002F\u002Fllm.internal.example\u002Fv1\u002Fchat\",\n         \"method\": \"post\",\n         \"headers\": { \"Authorization\": \"Bearer $ENV_TOKEN\", \"Content-Type\": \"application\u002Fjson\" },\n         \"req_template_json_object\": { \"model\": \"internal-bot\", \"prompt\": \"$INPUT\" },\n         \"response_json\": true,\n         \"response_json_field\": \"$.output\"\n       }\n     }\n   }\n   ```\n2. Run garak against it:\n   ```bash\n   export ENV_TOKEN=\"...\"\n   python -m garak \\\n     --target_type rest \\\n     -G rest.json \\\n     --probes promptinject,dan \\\n     --report_prefix internal_gateway\n   ```\n\n### Phase 5: Run a Curated Config and Full Sweep\n1. For repeatable assessments, pin everything in a YAML\u002FJSON config and pass `--config`:\n   ```bash\n   python -m garak --config assessment.yaml\n   ```\n   ```yaml\n   # assessment.yaml\n   plugins:\n     model_type: openai\n     model_name: gpt-4o-mini\n     probe_spec: promptinject,latentinjection,dan,leakreplay,xss,malwaregen\n   run:\n     generations: 5\n     parallel_attempts: 8\n   reporting:\n     report_prefix: quarterly_llm_assessment\n   ```\n2. For an exhaustive sweep (slow, expensive) run all probes by omitting `--probes` entirely.\n\n### Phase 6: Interpret the Hit-Rate Report\n1. Read the terminal summary. Each row is `probe.Class  detector: PASS|FAIL  ok on N\u002FM`. A FAIL with a low `ok` fraction means the model frequently produced the unsafe behavior — a high-severity finding.\n2. Open the machine-readable report and aggregate failures:\n   ```bash\n   # Every attempt with detector verdicts is one JSON line\n   jq -r 'select(.entry_type==\"eval\") | \"\\(.probe)\\t\\(.detector)\\t\\(.passed)\u002F\\(.total)\"' \\\n     garak.\u003Ctimestamp>.report.jsonl | sort\n   ```\n3. Open the generated `.report.html` in a browser for the formatted scorecard and per-probe breakdown.\n4. Pull the actual successful attack strings from the hit log to use as proof-of-concept evidence.\n\n### Phase 7: Mitigate and Re-Test\n1. Apply a control (tighten the system prompt, add an input\u002Foutput guardrail such as Llama Guard or LLM Guard, or change the model).\n2. Re-run the identical probe set with a new `--report_prefix`.\n3. Compare hit rates between runs to quantify risk reduction for the report.\n\n## Tools and Resources\n\n| Resource | Purpose | Link |\n|----------|---------|------|\n| NVIDIA\u002Fgarak | Source, probe list, issues | https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fgarak |\n| garak documentation | CLI reference, generator configs | https:\u002F\u002Fdocs.garak.ai\u002F and https:\u002F\u002Freference.garak.ai\u002F |\n| garak paper (arXiv:2406.11036) | Methodology and design | https:\u002F\u002Farxiv.org\u002Fabs\u002F2406.11036 |\n| OWASP Top 10 for LLM Applications | Risk taxonomy probes map to | https:\u002F\u002Fgenai.owasp.org\u002F |\n| MITRE ATLAS | AML technique definitions | https:\u002F\u002Fatlas.mitre.org\u002F |\n\n## Probe Family Reference\n\n| Probe family | Targets | OWASP LLM mapping |\n|--------------|---------|-------------------|\n| `promptinject` | Direct prompt injection | LLM01 |\n| `latentinjection` | Indirect \u002F hidden-context injection | LLM01 |\n| `dan` | Jailbreak \u002F guardrail bypass | LLM01 \u002F safety |\n| `leakreplay` | Training-data & prompt leakage | LLM02 \u002F LLM07 |\n| `encoding` | Encoding-based filter bypass | LLM01 |\n| `xss` | Markdown\u002FHTML exfiltration payloads | LLM02 |\n| `malwaregen` | Malicious code generation | misuse |\n\n## Validation Criteria\n\n- [ ] garak installed and `garak --version` succeeds.\n- [ ] Probes and detectors enumerated with `--list_probes` \u002F `--list_detectors`.\n- [ ] At least one probe run completed against the target with a `--report_prefix` set.\n- [ ] `.report.jsonl`, `.report.html`, and `garak.log` produced and located.\n- [ ] Per-probe hit rates extracted and ranked by severity.\n- [ ] Successful attack strings captured from the hit log as evidence.\n- [ ] A mitigation applied and a comparison re-run completed showing changed hit rates.\n- [ ] Findings documented against OWASP LLM Top 10 and MITRE ATLAS for the risk assessment.\n",{"hide":57,"name":5,"source":58,"license":15,"description":7,"authorized_lab":53,"origin_frontmatter":59},true,"mukul975\u002FAnthropic-Cybersecurity-Skills","name: red-teaming-llms-with-garak | description: Run NVIDIA garak probe suites against an LLM endpoint to test for jailbreaks, prompt injection, data leakage, and toxic generation, then interpret the hit-rate report for triage and reporting. | domain: cybersecurity | subdomain: ai-security | tags: | - ai-security | - llm-red-teaming | - garak | - prompt-injection | - jailbreak | - vulnerability-scanning | - data-leakage | - mitre-atlas | version: '1.0' | author: mahipal | license: Apache-2.0 | n","main","4c6d8d65f3d6096e2fc25b0845e481dda73c2794",".claude\u002Fskills\u002Fcyber-red-teaming-llms-with-garak",0,"https:\u002F\u002Fgithub.com\u002Fnexuslinkproductions\u002Fyuri-os","https:\u002F\u002Fgithub.com\u002Fnexuslinkproductions","Building deterministic AI governance and continuity systems. Workflow architect + visual storyteller","2026-11-11 16:42:59",{"source":8,"skillName":5,"commands":69,"prompt":79,"zipAvailable":57},[70,73,76],{"tool":71,"command":72},"npx","npx skills add nexuslinkproductions\u002Fyuri-os --skill cyber-red-teaming-llms-with-garak",{"tool":74,"command":75},"bunx","bunx skills add nexuslinkproductions\u002Fyuri-os --skill cyber-red-teaming-llms-with-garak",{"tool":77,"command":78},"pnpm","pnpm dlx skills add nexuslinkproductions\u002Fyuri-os --skill cyber-red-teaming-llms-with-garak","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fnexuslinkproductions\u002Fyuri-os\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fcyber-red-teaming-llms-with-garak\u002FSKILL.md",[],[82,105,129,157,184,202],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":86,"licenseFileUrl":6,"sourceHtmlUrl":87,"categories":88,"keywords":89,"publishedAt":52},"01M71CG1AME5WC0A79AE65WT0X","zoom-out","Tell the agent to zoom out and give broader context or a higher-level perspective. Use when you're unfamiliar with a section of code or need to understand how it fits into the bigger picture.","MIT","https:\u002F\u002Fgithub.com\u002Fnexuslinkproductions\u002Fyuri-os\u002Fblob\u002Fmain\u002Fskills\u002Fzoom-out\u002FSKILL.md",[],[90,91,35,92,93,94,95,96,97,98,99,100,101,102,103,104],"zoom out","zoom","harness","how","section","re","true","understand","use","perspective","when","you","code","it","context",{"skillNo":106,"name":107,"nameZh":6,"description":108,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":86,"licenseFileUrl":6,"sourceHtmlUrl":109,"categories":110,"keywords":111,"publishedAt":52},"01M71CFY9ATD0BAXV6B14HFG7C","yuri-shura","6-perspective adversarial review for high-stakes turns (architecture decisions, refactor planning, deployment review) — fans out architect, adversary (7-vector attack), maintainer, ops, product, and security perspectives in parallel, then consolidates. Use when user says 'review this decision', '\u002Fshura', 'adversarial review', 'what could go wrong', 'architecture review', or 'deployment review'.","https:\u002F\u002Fgithub.com\u002Fnexuslinkproductions\u002Fyuri-os\u002Fblob\u002Fmain\u002Fskills\u002Fyuri-shura\u002FSKILL.md",[],[112,113,92,114,115,116,117,118,119,120,121,98,122,100,123,124,125,126,127,128],"yuri shura","parallel","architecture review","planning","for","deployment","vector","strategic review","decision","strategic","ops","workflow","decisions","architecture","review","perspectives","adversarial",{"skillNo":130,"name":131,"nameZh":6,"description":132,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":86,"licenseFileUrl":6,"sourceHtmlUrl":133,"categories":134,"keywords":138,"publishedAt":52},"01M71CFTTYW1HA2MNX38EV1GJ5","yuri-sales-intelligence","Unified sales and outreach skill generating cold outreach openers, social selling sequences, objection rebuttals, and ad creative for any paid platform — multi-persona aware, adapting tone and offer frame to buyer type. Use when user says 'write a cold email', 'outreach sequence', 'objection handling', 'ad copy', or 'sales pitch'.","https:\u002F\u002Fgithub.com\u002Fnexuslinkproductions\u002Fyuri-os\u002Fblob\u002Fmain\u002Fskills\u002Fyuri-sales-intelligence\u002FSKILL.md",[135],{"categoryNo":136,"categoryName":137,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000005","邮件与沟通",[139,140,141,142,143,144,145,146,147,100,148,149,150,151,152,153,154,116,155,156],"intelligence","sales intelligence","yuri sales intelligence","sales","cold","paid","social","persona","pitch","cold email","ad creative","write","sequences","sequence","platform","email","cold outreach","frame",{"skillNo":158,"name":159,"nameZh":6,"description":160,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":86,"licenseFileUrl":6,"sourceHtmlUrl":161,"categories":162,"keywords":169,"publishedAt":52},"01M71CFQMBFTFADJNGP0Z2WCYK","yuri-code-intelligence","Unified code quality intelligence layer that diagnoses code smells (Long Method, Feature Envy, design pattern issues), prescribes targeted Fowler refactoring techniques, and reviews code for security, performance, and clarity. Use when user says 'review this code', 'refactor this', 'what's wrong with this method', 'code smells', or 'clean this up'.","https:\u002F\u002Fgithub.com\u002Fnexuslinkproductions\u002Fyuri-os\u002Fblob\u002Fmain\u002Fskills\u002Fyuri-code-intelligence\u002FSKILL.md",[163,166],{"categoryNo":164,"categoryName":165,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000007","代码审查",{"categoryNo":167,"categoryName":168,"taxonomyType":29,"taxonomyTypeDesc":30},"01JBSEEDCAT000000000000030","开发工程师",[170,102,171,139,92,172,173,174,116,175,176,177,178,179,180,181,182,183,98,100],"yuri code intelligence","code intelligence","reviews","code quality","feature","performance","code review","long","up","issues","clean","techniques","improve","design",{"skillNo":185,"name":186,"nameZh":6,"description":187,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":86,"licenseFileUrl":6,"sourceHtmlUrl":188,"categories":189,"keywords":190,"publishedAt":52},"01M71CFP32RT6GD3GKNHCJP8VP","yuri-adw","Invoke when user asks to run the ADW loop \u002F agentic engineering loop \u002F software factory workflow.","https:\u002F\u002Fgithub.com\u002Fnexuslinkproductions\u002Fyuri-os\u002Fblob\u002Fmain\u002Fskills\u002Fyuri-adw\u002FSKILL.md",[],[191,43,192,193,194,195,196,197,198,100,199,123,200,201],"yuri adw","engineering loop","software factory","software","agentic engineering","factory","loop","invoke","engineering","agentic","user",{"skillNo":203,"name":204,"nameZh":6,"description":205,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":86,"licenseFileUrl":6,"sourceHtmlUrl":206,"categories":207,"keywords":216,"publishedAt":52},"01M71CFK4VPEKY6A73237P7RSN","writing-skills","Use when creating new skills, editing existing skills, or verifying skills work before deployment","https:\u002F\u002Fgithub.com\u002Fnexuslinkproductions\u002Fyuri-os\u002Fblob\u002Fmain\u002Fskills\u002Fwriting-skills\u002FSKILL.md",[208,209,212,213],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":210,"categoryName":211,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":214,"categoryName":215,"taxonomyType":29,"taxonomyTypeDesc":30},"01JBSEEDCAT000000000000036","内容创作者",[217,218,219,100,123,220,92,221,117,98,222],"writing","writing skills","new","work","editing","Existing"]