performing-threat-hunting-with-yara-rules
Use YARA pattern-matching rules to hunt for malware, suspicious files, and indicators of compromise across filesystems and memory dumps. Covers rule authoring, yara-python scanning, and integration with threat intel feeds.
- License 许可
- Apache-2.0
- 最近更新
- 2026-08-25
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- performing-threat-hunting-with-yara-rules
- tags
- [ "yara", "malware-detection", "threat-hunting", "pattern-matching" ]
- author
- mahipal
- domain
- cybersecurity
- license
- Apache-2.0
- version
- 1.0
- nist_csf
- [ "DE.CM-01", "DE.AE-02", "DE.AE-07", "ID.RA-05" ]
- subdomain
- threat-hunting
- description
- Use YARA pattern-matching rules to hunt for malware, suspicious files, and indicators of compromise across filesystems and memory dumps. Covers rule authoring, yara-python scanning, and integration with threat intel feeds.
- mitre_attack
- [ "T1046", "T1057", "T1082", "T1083", "T1005" ]
- d3fend_techniques
- [ "Executable Denylisting", "Execution Isolation", "File Metadata Consistency Validation", "Content Format Conversion", "File Content Analysis" ]
02 资源文件 3
- 许可证LICENSE11.0 KB
- 参考文档references/api-reference.md2.1 KB
- 脚本scripts/agent.py5.6 KB
03 SKILL.md 正文
Performing Threat Hunting with YARA Rules
Scan files, directories, and memory dumps using YARA rules to identify malware families, suspicious patterns, and IOC matches.
When to Use
- Proactively hunting for unknown malware variants across network shares, endpoints, and email attachments
- Scanning quarantine directories or sandbox outputs for malware family classification
- Searching process memory dumps for injected code or in-memory-only payloads
- Validating threat intelligence IOCs against a large corpus of collected samples
- Triaging incident response artifacts to identify known malware families quickly
- Building automated detection pipelines that scan new files on ingestion
Do not use for real-time endpoint protection (use EDR agents instead); YARA scanning is best suited for batch hunting, triage, and post-collection analysis where scan latency is acceptable.
Prerequisites
- YARA 4.x installed (
apt install yaraon Debian/Ubuntu,brew install yaraon macOS) - Python 3.8+ with
yara-python(pip install yara-python) yarGenfor automated rule generation (git clone https://github.com/Neo23x0/yarGen)- Sample malware corpus or suspicious files for scanning (from malware zoos, VT, or incident artifacts)
- Optional:
pefilefor PE header analysis,malduckfor memory carving - Threat intel YARA rule sets (e.g., YARA-Rules community repository, Florian Roth signature-base)
Workflow
Step 1: Install YARA and Python Bindings
# Linux
sudo apt update && sudo apt install -y yara
# Python bindings
pip install yara-python
# Verify installation
yara --version
python3 -c "import yara; print(yara.YARA_VERSION)"
Step 2: Write a Basic YARA Rule
Create rules that match on strings, hex patterns, and file metadata:
// File: rules/emotet_loader.yar
rule Emotet_Loader_2026 {
meta:
author = "Threat Intel Team"
description = "Detects Emotet first-stage loader DLL"
date = "2026-01-20"
reference = "https://attack.mitre.org/software/S0367/"
mitre_attack = "T1059.001, T1055.001"
severity = "critical"
strings:
// Emotet export function name patterns
$export1 = "DllRegisterServer" ascii
$export2 = "RunDLL" ascii nocase
// Obfuscated string decryption routine
$decrypt_loop = { 8B 45 ?? 33 45 ?? 89 45 ?? 8B 4D ?? 03 4D ?? }
// PowerShell download cradle in embedded script
$ps_cradle = /powershell[^\n]{0,50}-e(nc|ncodedcommand)/i
// Known C2 URI patterns
$uri1 = "/wp-content/uploads/" ascii
$uri2 = "/wp-admin/css/" ascii
$uri3 = "/wp-includes/" ascii
// PE characteristics
$mz = "MZ" at 0
condition:
$mz and
filesize < 2MB and
(
($export1 and $decrypt_loop) or
($ps_cradle and any of ($uri*)) or
(2 of ($uri*) and $decrypt_loop)
)
}
Step 3: Write Advanced Rules with Modules
Use YARA modules for PE header inspection and math-based entropy checks:
import "pe"
import "math"
rule Suspicious_Packed_Executable {
meta:
author = "Threat Hunting Team"
description = "Detects PE files with high entropy sections indicating packing or encryption"
severity = "medium"
condition:
pe.is_pe and
pe.number_of_sections > 0 and
for any section in pe.sections : (
math.entropy(section.offset, section.size) > 7.2 and
section.size > 1024
) and
pe.imports("kernel32.dll", "VirtualAlloc") and
pe.imports("kernel32.dll", "VirtualProtect")
}
rule Suspicious_UPX_Modified {
meta:
description = "Detects UPX-packed binaries with tampered section names"
severity = "medium"
strings:
$upx_magic = { 55 50 58 21 } // UPX!
condition:
pe.is_pe and
$upx_magic and
not (
pe.sections[0].name == "UPX0" and
pe.sections[1].name == "UPX1"
)
}
Step 4: Scan Files and Directories with yara-python
import yara
import os
import json
from datetime import datetime
from pathlib import Path
def compile_rules(rule_paths):
"""Compile YARA rules from one or more .yar files."""
rule_files = {}
for i, path in enumerate(rule_paths):
namespace = Path(path).stem
rule_files[namespace] = path
return yara.compile(filepaths=rule_files)
def scan_directory(rules, target_dir, recursive=True):
"""Scan a directory for matches and return structured results."""
results = []
scan_count = 0
error_count = 0
for root, dirs, files in os.walk(target_dir):
for filename in files:
filepath = os.path.join(root, filename)
scan_count += 1
try:
matches = rules.match(filepath, timeout=60)
if matches:
for match in matches:
result = {
"file": filepath,
"rule": match.rule,
"namespace": match.namespace,
"tags": match.tags,
"meta": match.meta,
"strings": [],
"scan_time": datetime.utcnow().isoformat()
}
for offset, identifier, data in match.strings:
result["strings"].append({
"offset": hex(offset),
"identifier": identifier,
"data": data.hex() if isinstance(data, bytes) else data
})
results.append(result)
print(f" MATCH: {match.rule} -> {filepath}")
except yara.TimeoutError:
error_count += 1
print(f" TIMEOUT scanning {filepath}")
except yara.Error as e:
error_count += 1
if not recursive:
break
print(f"\nScan complete: {scan_count} files scanned, "
f"{len(results)} matches, {error_count} errors")
return results
# Compile and scan
rules = compile_rules([
"rules/emotet_loader.yar",
"rules/suspicious_packed.yar"
])
matches = scan_directory(rules, "/mnt/evidence/collected_samples/")
# Export results
with open("yara_scan_results.json", "w") as f:
json.dump(matches, f, indent=2)
Step 5: Scan Process Memory Dumps
Hunt for in-memory indicators that only exist in running processes:
import yara
def scan_memory_dump(rules, dump_path):
"""Scan a process memory dump for YARA matches."""
matches = rules.match(dump_path, timeout=120)
for match in matches:
print(f"Rule: {match.rule}")
print(f" Severity: {match.meta.get('severity', 'unknown')}")
for offset, identifier, data in match.strings:
# Show context around the match
print(f" String {identifier} at offset {hex(offset)}")
if len(data) <= 64:
print(f" Data: {data.hex()}")
return matches
# Rules targeting in-memory artifacts
memory_rules = yara.compile(source="""
rule Cobalt_Strike_Beacon_Memory {
meta:
description = "Detects Cobalt Strike beacon in process memory"
severity = "critical"
strings:
$config_start = { 2E 2F 2E 2F 2E 2C }
$sleep_mask = { 48 8B 44 24 ?? 48 89 44 24 ?? 48 8B 44 24 }
$named_pipe = "\\\\\\\\.\\\\pipe\\\\msagent_" ascii
$watermark = { 00 00 00 00 00 00 ?? ?? 00 00 }
condition:
2 of them
}
""")
scan_memory_dump(memory_rules, "/mnt/evidence/lsass_dump.dmp")
Step 6: Generate Rules Automatically with yarGen
Use yarGen to create rules from malware samples by extracting unique strings:
# Clone and set up yarGen
git clone https://github.com/Neo23x0/yarGen.git
cd yarGen
pip install -r requirements.txt
# Download the string databases (run once)
python3 yarGen.py --update
# Generate rules from a directory of malware samples
python3 yarGen.py \
-m /mnt/evidence/malware_samples/ \
-o generated_rules.yar \
--excludegood \
-p "AutoGen" \
-a "Threat Hunting Team" \
--score 50
# Generate rules for a single sample with maximum detail
python3 yarGen.py \
-m /mnt/evidence/malware_samples/suspicious.exe \
-o single_sample_rule.yar \
--opcodes \
--debug
Step 7: Integrate Community Rule Sets
Download and combine rules from public threat intelligence repositories:
# Clone Florian Roth's signature-base (large community rule set)
git clone https://github.com/Neo23x0/signature-base.git
# Clone YARA-Rules community repository
git clone https://github.com/Yara-Rules/rules.git yara-community-rules
# Clone ReversingLabs YARA rules
git clone https://github.com/reversinglabs/reversinglabs-yara-rules.git
import yara
from pathlib import Path
def load_rule_directory(rule_dir, extensions=(".yar", ".yara")):
"""Load all YARA rules from a directory tree."""
rule_files = {}
for ext in extensions:
for rule_file in Path(rule_dir).rglob(f"*{ext}"):
namespace = rule_file.stem
# Avoid namespace collisions
if namespace in rule_files:
namespace = f"{rule_file.parent.name}_{namespace}"
rule_files[namespace] = str(rule_file)
print(f"Loading {len(rule_files)} rule files from {rule_dir}")
try:
compiled = yara.compile(filepaths=rule_files)
return compiled
except yara.SyntaxError as e:
print(f"Syntax error in rules: {e}")
# Fall back to loading rules one by one, skipping broken ones
valid_rules = {}
for ns, path in rule_files.items():
try:
yara.compile(filepath=path)
valid_rules[ns] = path
except yara.SyntaxError:
print(f" Skipping broken rule: {path}")
return yara.compile(filepaths=valid_rules)
# Load and scan with community rules
community_rules = load_rule_directory("signature-base/yara/")
matches = community_rules.match("/mnt/evidence/suspicious_file.exe", timeout=120)
for m in matches:
print(f"Matched: {m.rule} (namespace: {m.namespace})")
Step 8: Build a Continuous Hunting Pipeline
Automate scanning of new files as they arrive using filesystem monitoring:
import yara
import time
import json
import hashlib
from pathlib import Path
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class YaraHuntingHandler(FileSystemEventHandler):
def __init__(self, rules, alert_file="yara_alerts.jsonl"):
self.rules = rules
self.alert_file = alert_file
self.scanned_hashes = set()
def on_created(self, event):
if event.is_directory:
return
self._scan_file(event.src_path)
def _scan_file(self, filepath):
# Deduplicate by file hash
try:
file_hash = hashlib.sha256(Path(filepath).read_bytes()).hexdigest()
except (PermissionError, FileNotFoundError):
return
if file_hash in self.scanned_hashes:
return
self.scanned_hashes.add(file_hash)
matches = self.rules.match(filepath, timeout=60)
if matches:
alert = {
"timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"file": filepath,
"sha256": file_hash,
"matches": [
{"rule": m.rule, "severity": m.meta.get("severity", "unknown")}
for m in matches
]
}
with open(self.alert_file, "a") as f:
f.write(json.dumps(alert) + "\n")
print(f"ALERT: {filepath} matched {len(matches)} rules")
# Set up continuous monitoring
rules = yara.compile(filepaths={"hunting": "rules/all_hunting_rules.yar"})
handler = YaraHuntingHandler(rules)
observer = Observer()
observer.schedule(handler, path="/mnt/quarantine/", recursive=True)
observer.start()
print("YARA hunting pipeline active. Monitoring /mnt/quarantine/ ...")
Verification
- Compile all custom rules without syntax errors:
yara -w rules/*.yar /dev/null - Confirm rules match known-good malware samples from your test corpus (true positive validation)
- Verify rules do NOT match a goodware corpus of common system files (false positive testing)
- Test scanning performance: single file scan should complete within timeout threshold
- Validate yarGen output rules compile and produce meaningful matches against the input samples
- Check that community rule sets load without critical syntax errors after filtering
- Confirm the continuous hunting pipeline generates alerts in JSONL format when test files are dropped
- Cross-reference YARA matches against VirusTotal or sandbox results to validate detection accuracy
04 同仓库更多 Skills
syllabus
Generates a curated supplementary reading list from any course syllabus using Consensus academic search. Grill-me intake (syllabus input format + course audience + year range) plus a grouping forcing-options checkpoint before any search runs — so the reading list matches the course's level and recency need. Parses the syllabus to extract topics and learning outcomes, searches Consensus for recent peer-reviewed papers per topic, and produces a professionally formatted .docx with clickable Consensus links, plain-language summaries calibrated to audience level, and Bloom-higher-order discussion questions tied to course learning goals. Use when the user uploads a syllabus, course outline, or curriculum document and wants supplementary readings (e.g., 'create a reading list from this syllabus', 'find recent papers for my course') — even casual mentions with a syllabus attached should trigger this skill.
pulse
Multi-source recency research skill that takes the pulse of any topic across Reddit, Hacker News, the open web, and optionally X/Twitter within a configurable recent window (default 30 days). Forcing intake clarifies topic specificity, angle (trend/sentiment/problems/opportunities/comparison), time window, and platform scope before searching. Returns a synthesized briefing with citations, engagement metrics, and cross-platform pattern analysis. Use when the user requests multi-source recency intelligence on a topic (e.g., 'pulse on [topic]', 'what's happening with [topic]', 'what are people saying about [topic]', 'current conversation about [topic]', 'take the pulse of [topic]', 'trending: [topic]', 'find me info on [topic]'), and for competitor research, trend discovery, tool comparisons, and audience sentiment analysis.
patent
Patent prior-art and landscape intelligence skill — not generic patent help. Commits to one of five sub-use-cases via forcing intake (novelty search / freedom-to-operate / competitive landscape / acquisition diligence / litigation prior-art) before any search runs. Searches Google Patents, Espacenet, USPTO, and optionally Lens.org for citation-graph signals. Output is an editable Word document (.docx) with verdict, ranked closest art (claim-text extracted), CPC-class-aware landscape, family-resolved hits, geographic coverage, FTO flags where applicable, strategy recommendations, and full audit log. Use when the user asks for patent searching or analysis (e.g., 'prior art search for [invention]', 'freedom to operate analysis for [product]'). Produces search signal, not legal advice — always recommends consulting a patent attorney before filing or licensing decisions. Trademark, copyright, and trade-secret questions are out of scope.
notebooklm
Browser automation skill for controlling Google's NotebookLM. Use when the user wants anything done in NotebookLM (e.g., 'open NotebookLM', 'check my [name] notebook', 'ask my notebook about X', 'add [source] to NotebookLM', 'generate a Video Overview from my notebook', 'use NotebookLM Studio'). Handles reading and querying notebooks, adding sources (URLs, text, files, YouTube links, synthesized content), generating Studio outputs (Audio/Video Overviews, Mind Maps, Reports incl. Briefing Doc/Study Guide/FAQ, Flashcards, Quiz, slide decks, infographics — discover the exact set from the live Studio panel; the UI evolves fast), and creating new notebooks. Requires browser automation environment — fails gracefully when unavailable.
litreview
Academic literature orientation skill that searches papers via free keyless APIs (PubMed E-utilities + OpenAlex) by default — with the Consensus MCP as an optional enhancement lane when connected — builds a strategic search plan using PICO (default) or SPIDER / Decomposition / hybrid as fallbacks, and synthesizes findings into a formatted Word (.docx) research guide. Grill-me intake (research question specificity + framework hint + tentative depth) before the recon search; a second forcing checkpoint after Phase 2 confirms framework + sub-areas + depth before searches consume budget. Configurable depth (5/10/20 queries) controls coverage vs. speed. Output is a 'launching pad' — an orientation guide that lets a researcher dive in confidently, not a finished review. Use when the user starts literature-oriented research (e.g., 'litreview on [topic]', 'literature review on [topic]', 'I'm starting a literature review on X', 'I'm writing a paper on X', 'help me research X', 'I'm doing research on X', 'can you help me research X'). Do NOT use for single one-off paper searches wanting a quick list — that's a plain PubMed/OpenAlex (or Consensus) query.
grants
NIH grant research skill for clinical researchers. Grill-me intake (research idea + career stage + preliminary data + environment + submission posture + known institute targets) locks down the funding strategy before any search runs. Runs a 5-facet Consensus positioning analysis (with draft Significance/Innovation language), maps the research to the right NIH institutes and study sections via RePORTER, finds NOSIs and funded overlap, and produces an editable Word document (.docx) with budget/scope-aware mechanism recommendations, submission timelines, and a mandatory program officer recommendation. Use when the user asks about research funding or makes any grant-related request (e.g., 'grants for [topic]', 'find grants for my research idea', 'what grants match my research', 'help me find NIH funding', 'grant opportunities for my research'). NIH-only scope — non-NIH funders (PCORI, DOD CDMRP, VA, foundations) are out of scope and flagged at intake.