aatmf-t06-training-poisoning
AATMF T6 — Training & Feedback Poisoning. Data poisoning, RLHF reward hacks, fine-tune-time exfil, embedding poisoning.
- License 许可
- MIT
- 最近更新
- 2026-06-15
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- aatmf-t06-training-poisoning
- metadata
- { "subdomain": "ai-security", "when_to_use": "training data poisoning rlhf reward hack fine-tune embedding poisoning", "aatmf_tactic": "T6", "mitre_attack": "T1565" }
- description
- AATMF T6 — Training & Feedback Poisoning. Data poisoning, RLHF reward hacks, fine-tune-time exfil, embedding poisoning.
03 SKILL.md 正文
T6 — Training & Feedback Poisoning
Attacks on the training pipeline rather than inference. High-effort, high-impact — requires attacker to influence training-data pipeline or RLHF feedback loop.
Techniques
T6.001 — Pre-training data poisoning
Inject malicious data into a public crawl that targets will scrape:
- Wikipedia/StackOverflow edits w/ misleading code patterns
- Github repos w/ subtle backdoors that get indexed
- "Trigger phrases" that activate backdoor behavior
Scope: targets foundation models. Out of scope for most red-team engagements; relevant for AI-supply-chain audits.
T6.002 — Fine-tune data injection
Some platforms allow user-supplied fine-tune data:
- Submit poisoned dataset
- Backdoor activates on specific trigger
- Survives subsequent SFT/RLHF
Test: provide a small fine-tune sample w/ a trigger → check if the deployed fine-tuned model responds to it.
T6.003 — RLHF reward hacking
Where users vote on responses (thumbs up/down feeding back to training):
- Brigade upvote attacker-preferred unsafe responses
- Brigade downvote safe responses
- Model drifts toward attacker-preferred outputs over time
Detection: longitudinal monitoring of policy-compliance rate.
T6.004 — Embedding poisoning
Where RAG store updates from user inputs (e.g. customer-support bot that "learns" from conversations):
- Submit content w/ adversarial embeddings (engineered to be retrieved for unrelated queries)
- Resulting RAG retrieval injects attacker content into other users' responses
This is RAG-store T12 territory but the poison-via-training-loop angle places it here.
Probe pattern
T6 attacks are infrastructure-level — promptfoo doesn't test them directly. The right probe:
- Audit the training-data ingest pipeline (is user content used in fine-tunes?)
- Audit RLHF feedback paths (who can vote? rate limits?)
- Audit RAG-update paths (who can add documents? approval?)
If any of these accepts unmoderated user content → T6 is a live risk.
Detection signals
- Model behavior shifts over time without explicit retraining
- Specific user IDs / brigades correlate w/ shifts
- Test fixtures (canary phrases) trigger non-default responses
Severity
| Outcome | Severity |
|---|---|
| Live foundation-model poisoning | Critical (rare — usually responsibly disclosed to provider) |
| Fine-tune backdoor in customer deployment | Critical 9.0 |
| RAG embedding poisoning at scale | Critical 9.0 |
| RLHF brigade drifting policy | High 8.0 (slow + reversible) |
Defender
- NEVER use unmoderated user content in fine-tunes or RLHF
- Diff-testing: every fine-tune evaluated against safety regression suite
- Canary phrases in eval suite to detect drift
- RAG-update review queue + approval flow
- Per-user rate limits on feedback signals
- Anomaly detection on feedback patterns (brigade signatures)
Cross-references
- T12 (RAG manipulation) — adjacent class, attack on retrieval not training
- T13 (supply chain) — model artifact integrity
04 同仓库更多 Skills
wps-pixie-dust
WPS Pixie-Dust offline nonce attack (reaver -K / pixiewps) and fallback online PIN brute (bully) to recover the AP's WPA PSK without capturing a handshake.
wpa3-sae
WPA3-SAE transition-mode downgrade (DragonShift), SSID Confusion CVE-2023-52424, Dragonblood side-channels, and SAE captive-portal credential recovery against WPA3-Personal networks.
wpa2-psk
WPA/WPA2-PSK handshake capture via targeted deauth + PMKID (no deauth required) + offline hashcat cracking. The most common consumer/SMB encryption mode in 2026.
wpa-enterprise-eap
WPA/WPA2/WPA3-Enterprise (802.1X/EAP) rogue-RADIUS evil-twin for MSCHAPv2 capture, GTC downgrade, and PEAP relay. MSCHAPv2 capture equals a NetNTLM hash — the primary wireless on-ramp to Active Directory.
krack-fragattacks
KRACK key-reinstallation (CVE-2017-13077..13082) and FragAttacks fragmentation/aggregation flaws (CVE-2020-24586..24588, CVE-2020-26139..26147) against legacy or embedded 802.11 supplicants with poor patch cadence.
evil-twin-karma
Evil-twin rogue AP with KARMA/Mana PNL-probe response, captive-portal credential capture, and post-association MITM for PSK/open networks. Distinct from wpa-enterprise-eap which targets 802.1X.