detecting-network-anomalies-with-zeek
Deploy and configure Zeek (formerly Bro) to passively analyze network traffic, generate structured connection/DNS/HTTP/SSL/file logs, detect anomalous behavior, and write custom scripts for organization-specific threats. Use for passive monitoring at network choke points, feeding SIEM/threat hunting with protocol metadata, or retrospective log analysis during incident response; not a substitute for inline IDS/IPS or host agents.
- License 许可
- Apache-2.0
- 最近更新
- 2026-08-25
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- detecting-network-anomalies-with-zeek
- tags
- [ "network-security", "zeek", "network-monitoring", "anomaly-detection", "threat-hunting" ]
- author
- mahipal
- domain
- cybersecurity
- license
- Apache-2.0
- version
- 1.0
- nist_csf
- [ "PR.IR-01", "DE.CM-01", "ID.AM-03", "PR.DS-02" ]
- subdomain
- network-security
- description
- Deploy and configure Zeek (formerly Bro) to passively analyze network traffic, generate structured connection/DNS/HTTP/SSL/file logs, detect anomalous behavior, and write custom scripts for organization-specific threats. Use for passive monitoring at network choke points, feeding SIEM/threat hunting with protocol metadata, or retrospective log analysis during incident response; not a substitute for inline IDS/IPS or host agents.
- mitre_attack
- [ "T1046", "T1040", "T1557", "T1071" ]
02 资源文件 3
- 许可证LICENSE11.0 KB
- 参考文档references/api-reference.md2.5 KB
- 脚本scripts/agent.py11.1 KB
03 SKILL.md 正文
Detecting Network Anomalies with Zeek
When to Use
- Deploying passive network security monitoring at key network choke points for continuous visibility
- Generating structured connection, DNS, HTTP, SSL, and file transfer logs for SIEM ingestion and threat hunting
- Writing custom Zeek scripts to detect organization-specific threats, policy violations, or beaconing behavior
- Performing retrospective analysis on network metadata to investigate security incidents
- Complementing IDS solutions with protocol-level metadata analysis that signature-based tools may miss
Do not use as a replacement for inline IDS/IPS that can actively block traffic, for monitoring encrypted payloads without TLS inspection, or on endpoints where host-based agents are more appropriate.
Prerequisites
- Zeek 6.0+ installed from source or package manager (
zeek --version) - Network interface configured on a span port, network tap, or virtual switch mirror for passive capture
- Sufficient disk storage for log files (estimate 1-5 GB/day per 100 Mbps of monitored traffic)
- Familiarity with Zeek's scripting language for writing custom detections
- Log aggregation system (Splunk, Elastic, Graylog) for centralized analysis
Workflow
Step 1: Install and Configure Zeek
# Install Zeek on Ubuntu/Debian
sudo apt install -y zeek
# Or install from source for latest version
git clone --recursive https://github.com/zeek/zeek
cd zeek && ./configure --prefix=/opt/zeek && make -j$(nproc) && sudo make install
export PATH=/opt/zeek/bin:$PATH
# Configure the monitoring interface
sudo vi /opt/zeek/etc/node.cfg
# /opt/zeek/etc/node.cfg
[zeek]
type=standalone
host=localhost
interface=eth1
# Configure local network definitions
sudo vi /opt/zeek/etc/networks.cfg
# /opt/zeek/etc/networks.cfg
10.0.0.0/8 Internal
172.16.0.0/12 Internal
192.168.0.0/16 Internal
# Disable NIC offloading for accurate packet capture
sudo ethtool -K eth1 rx off tx off gro off lro off tso off gso off
# Deploy Zeek
sudo zeekctl deploy
# Verify Zeek is running
sudo zeekctl status
Step 2: Understand and Navigate Zeek Logs
# Zeek generates structured log files in /opt/zeek/logs/current/
ls /opt/zeek/logs/current/
# Key log files:
# conn.log - All network connections (TCP, UDP, ICMP)
# dns.log - DNS queries and responses
# http.log - HTTP requests and responses
# ssl.log - SSL/TLS handshake details
# files.log - File transfers observed on the network
# notice.log - Alerts from Zeek detection scripts
# weird.log - Protocol anomalies and errors
# x509.log - X.509 certificate details
# smtp.log - SMTP email transactions
# ssh.log - SSH connection details
# View connection log with zeek-cut for column selection
cat /opt/zeek/logs/current/conn.log | zeek-cut ts id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes
# View DNS log
cat /opt/zeek/logs/current/dns.log | zeek-cut ts id.orig_h query qtype_name answers
# View HTTP log
cat /opt/zeek/logs/current/http.log | zeek-cut ts id.orig_h host uri method status_code user_agent
Step 3: Write Custom Detection Scripts
# Create a custom detection script directory
sudo mkdir -p /opt/zeek/share/zeek/site/custom-detections
Create a script for detecting DNS tunneling:
# /opt/zeek/share/zeek/site/custom-detections/dns-tunneling.zeek
@load base/frameworks/notice
module DNSTunneling;
export {
redef enum Notice::Type += {
DNS_Tunneling_Detected,
DNS_Long_Query
};
# Threshold: number of unique queries per source in time window
const query_threshold: count = 200 &redef;
const time_window: interval = 5min &redef;
const max_query_length: count = 50 &redef;
}
# Track query counts per source IP
global dns_query_counts: table[addr] of count &create_expire=5min &default=0;
event dns_request(c: connection, msg: dns_msg, query: string, qtype: count, qclass: count)
{
local src = c$id$orig_h;
# Check for unusually long domain queries (base64-encoded data)
if ( |query| > max_query_length )
{
NOTICE([
$note=DNS_Long_Query,
$msg=fmt("Unusually long DNS query from %s: %s (%d chars)", src, query, |query|),
$src=src,
$identifier=cat(src, query)
]);
}
# Track query volume per source
dns_query_counts[src] += 1;
if ( dns_query_counts[src] == query_threshold )
{
NOTICE([
$note=DNS_Tunneling_Detected,
$msg=fmt("Possible DNS tunneling: %s sent %d queries in %s", src, query_threshold, time_window),
$src=src,
$identifier=cat(src)
]);
}
}
Create a script for detecting beaconing:
# /opt/zeek/share/zeek/site/custom-detections/beacon-detection.zeek
@load base/frameworks/notice
@load base/frameworks/sumstats
module BeaconDetection;
export {
redef enum Notice::Type += {
Possible_Beaconing
};
const beacon_threshold: count = 50 &redef;
const observation_window: interval = 1hr &redef;
}
event zeek_init()
{
local r1 = SumStats::Reducer(
$stream="beacon.connections",
$apply=set(SumStats::SUM)
);
SumStats::create([
$name="detect-beaconing",
$epoch=observation_window,
$reducers=set(r1),
$threshold_val(key: SumStats::Key, result: SumStats::Result) = {
return result["beacon.connections"]$sum;
},
$threshold=beacon_threshold + 0.0,
$threshold_crossed(key: SumStats::Key, result: SumStats::Result) = {
NOTICE([
$note=Possible_Beaconing,
$msg=fmt("Possible beaconing: %s made %d connections in %s",
key$str, result["beacon.connections"]$sum, observation_window),
$identifier=key$str
]);
}
]);
}
event connection_state_remove(c: connection)
{
if ( c$id$resp_h !in Site::local_nets )
{
local key = fmt("%s->%s:%d", c$id$orig_h, c$id$resp_h, c$id$resp_p);
SumStats::observe("beacon.connections", [$str=key], [$num=1]);
}
}
Step 4: Load Custom Scripts and Deploy
# Add custom scripts to local.zeek
sudo tee -a /opt/zeek/share/zeek/site/local.zeek << 'EOF'
# Custom detection scripts
@load custom-detections/dns-tunneling.zeek
@load custom-detections/beacon-detection.zeek
# Enable additional protocol analyzers
@load protocols/ftp/software
@load protocols/http/software
@load protocols/smtp/software
@load protocols/ssh/detect-bruteforcing
@load protocols/ssl/validate-certs
@load protocols/ssl/log-hostcerts-only
@load protocols/dns/detect-external-names
# Enable file extraction
@load frameworks/files/extract-all-files
# Enable Intel framework for threat intelligence
@load frameworks/intel/seen
@load frameworks/intel/do_notice
EOF
# Reload Zeek configuration
sudo zeekctl deploy
# Verify scripts loaded without errors
sudo zeekctl diag
Step 5: Threat Hunting Queries on Zeek Logs
# Find long-duration connections (possible C2)
cat /opt/zeek/logs/current/conn.log | zeek-cut ts id.orig_h id.resp_h id.resp_p duration | \
awk '$5 > 3600 {print $0}' | sort -t$'\t' -k5 -rn | head -20
# Find connections with high data transfer volumes
cat /opt/zeek/logs/current/conn.log | zeek-cut ts id.orig_h id.resp_h orig_bytes resp_bytes | \
awk '$4 > 100000000 || $5 > 100000000 {print $0}'
# Identify rare user agents (potential malware)
cat /opt/zeek/logs/current/http.log | zeek-cut user_agent | sort | uniq -c | sort -n | head -20
# Find self-signed or expired certificates
cat /opt/zeek/logs/current/ssl.log | zeek-cut ts id.orig_h id.resp_h server_name validation_status | \
grep -v "ok"
# Detect DNS queries to newly registered domains (DGA patterns)
cat /opt/zeek/logs/current/dns.log | zeek-cut ts id.orig_h query | \
awk -F'\t' '{n=split($3,a,"."); if(length(a[n-1]) > 10) print $0}'
# Find SSH brute force attempts
cat /opt/zeek/logs/current/ssh.log | zeek-cut ts id.orig_h id.resp_h auth_success | \
grep "F" | awk '{print $2}' | sort | uniq -c | sort -rn | head -10
# Identify unusual port usage
cat /opt/zeek/logs/current/conn.log | zeek-cut id.resp_p proto service | \
sort | uniq -c | sort -rn | head -50
Step 6: Integrate with SIEM and Set Up Alerting
# Configure JSON log output for SIEM ingestion
sudo tee /opt/zeek/share/zeek/site/json-logs.zeek << 'EOF'
@load policy/tuning/json-logs.zeek
redef LogAscii::use_json = T;
EOF
# Forward logs to Elastic via Filebeat
# /etc/filebeat/filebeat.yml
sudo tee /etc/filebeat/filebeat.yml << 'EOF'
filebeat.inputs:
- type: log
enabled: true
paths:
- /opt/zeek/logs/current/*.log
json.keys_under_root: true
json.add_error_key: true
fields:
source: zeek
fields_under_root: true
output.elasticsearch:
hosts: ["https://elastic-siem:9200"]
index: "zeek-%{+yyyy.MM.dd}"
username: "elastic"
password: "${ES_PASSWORD}"
EOF
sudo systemctl enable --now filebeat
# Set up log rotation
sudo tee /etc/cron.d/zeek-logrotate << 'EOF'
0 0 * * * root /opt/zeek/bin/zeekctl cron
EOF
# Monitor Zeek health
sudo zeekctl status
sudo zeekctl netstats
Key Concepts
| Term | Definition |
|---|---|
| Network Security Monitor | Passive analysis tool that observes network traffic and generates structured metadata logs without altering or blocking traffic flow |
| Zeek Script | Event-driven scripts written in Zeek's domain-specific language that process network events and generate notices, logs, and metrics |
| Connection Log (conn.log) | Core Zeek log recording every observed connection with source/destination IPs, ports, protocol, duration, and byte counts |
| Notice Framework | Zeek subsystem for generating alerts when detection scripts identify suspicious activity, outputting to notice.log |
| SumStats Framework | Statistical analysis framework in Zeek for tracking metrics over time windows, enabling threshold-based detection of anomalies |
| Intel Framework | Zeek module for matching observed network indicators against threat intelligence feeds and generating alerts on matches |
Tools & Systems
- Zeek 6.0+: Open-source network security monitor generating comprehensive protocol-level logs from passive traffic analysis
- zeek-cut: Zeek utility for extracting specific columns from tab-separated Zeek log files for quick analysis
- zeekctl: Zeek management tool for deploying, monitoring, and managing Zeek instances across single or clustered deployments
- RITA (Real Intelligence Threat Analytics): Open-source tool that analyzes Zeek logs for beaconing, DNS tunneling, and other threat indicators
- Filebeat: Elastic agent for shipping Zeek JSON logs to Elasticsearch for centralized analysis and visualization
Common Scenarios
Scenario: Detecting Command-and-Control Beaconing in Enterprise Traffic
Context: A threat intelligence report indicates that a specific threat actor uses HTTPS beaconing with 60-second intervals to compromised hosts. The SOC team needs to analyze Zeek logs to identify any hosts exhibiting this pattern across the enterprise network carrying 2 Gbps of traffic.
Approach:
- Deploy Zeek on a network tap at the internet egress point with AF_PACKET for high-throughput capture
- Enable the custom beacon detection script with thresholds tuned for 60-second intervals over 1-hour observation windows
- Query conn.log for connections to external IPs with consistent duration and inter-connection timing: filter connections where the standard deviation of inter-arrival times is less than 5 seconds
- Cross-reference suspicious destination IPs against threat intelligence feeds loaded into Zeek's Intel framework
- Examine ssl.log for the associated TLS certificates -- check for self-signed certificates, unusual issuer names, or certificates with short validity periods
- Generate a notice for each identified beaconing source and feed into the SIEM for SOC triage
Pitfalls:
- Not tuning beacon detection thresholds for the environment, resulting in false positives from legitimate update services (Windows Update, AV updates)
- Failing to exclude CDN and cloud service provider IP ranges that naturally receive many repeat connections
- Running Zeek without sufficient CPU cores, causing packet drops on high-throughput links
- Not enabling JSON log output, making SIEM integration unnecessarily complex with custom parsers
Output Format
## Zeek Network Anomaly Detection Report
**Sensor**: zeek-sensor-01 (10.10.1.250)
**Monitoring Interface**: eth1 (span port from Core-SW1)
**Analysis Period**: 2024-03-15 00:00 to 2024-03-16 00:00 UTC
**Total Connections Logged**: 2,847,392
### Anomalies Detected
| Notice Type | Source | Destination | Details |
|-------------|--------|-------------|---------|
| DNS_Tunneling_Detected | 10.10.3.45 | 8.8.8.8 | 847 queries to suspect-domain.xyz in 5 min |
| Possible_Beaconing | 10.10.5.12 | 203.0.113.50:443 | 62 connections with 59.8s avg interval |
| SSL::Invalid_Server_Cert | 10.10.8.22 | 198.51.100.33:443 | Self-signed cert, CN=localhost |
| SSH::Password_Guessing | 45.33.32.156 | 10.10.20.11:22 | 487 failed attempts in 30 min |
### Recommendations
1. Isolate 10.10.3.45 and investigate for DNS tunneling malware
2. Block 203.0.113.50 at firewall and forensically image 10.10.5.12
3. Investigate self-signed TLS certificate on 198.51.100.33
4. Block 45.33.32.156 and enforce SSH key-only authentication
04 同仓库更多 Skills
syllabus
Generates a curated supplementary reading list from any course syllabus using Consensus academic search. Grill-me intake (syllabus input format + course audience + year range) plus a grouping forcing-options checkpoint before any search runs — so the reading list matches the course's level and recency need. Parses the syllabus to extract topics and learning outcomes, searches Consensus for recent peer-reviewed papers per topic, and produces a professionally formatted .docx with clickable Consensus links, plain-language summaries calibrated to audience level, and Bloom-higher-order discussion questions tied to course learning goals. Use when the user uploads a syllabus, course outline, or curriculum document and wants supplementary readings (e.g., 'create a reading list from this syllabus', 'find recent papers for my course') — even casual mentions with a syllabus attached should trigger this skill.
pulse
Multi-source recency research skill that takes the pulse of any topic across Reddit, Hacker News, the open web, and optionally X/Twitter within a configurable recent window (default 30 days). Forcing intake clarifies topic specificity, angle (trend/sentiment/problems/opportunities/comparison), time window, and platform scope before searching. Returns a synthesized briefing with citations, engagement metrics, and cross-platform pattern analysis. Use when the user requests multi-source recency intelligence on a topic (e.g., 'pulse on [topic]', 'what's happening with [topic]', 'what are people saying about [topic]', 'current conversation about [topic]', 'take the pulse of [topic]', 'trending: [topic]', 'find me info on [topic]'), and for competitor research, trend discovery, tool comparisons, and audience sentiment analysis.
patent
Patent prior-art and landscape intelligence skill — not generic patent help. Commits to one of five sub-use-cases via forcing intake (novelty search / freedom-to-operate / competitive landscape / acquisition diligence / litigation prior-art) before any search runs. Searches Google Patents, Espacenet, USPTO, and optionally Lens.org for citation-graph signals. Output is an editable Word document (.docx) with verdict, ranked closest art (claim-text extracted), CPC-class-aware landscape, family-resolved hits, geographic coverage, FTO flags where applicable, strategy recommendations, and full audit log. Use when the user asks for patent searching or analysis (e.g., 'prior art search for [invention]', 'freedom to operate analysis for [product]'). Produces search signal, not legal advice — always recommends consulting a patent attorney before filing or licensing decisions. Trademark, copyright, and trade-secret questions are out of scope.
notebooklm
Browser automation skill for controlling Google's NotebookLM. Use when the user wants anything done in NotebookLM (e.g., 'open NotebookLM', 'check my [name] notebook', 'ask my notebook about X', 'add [source] to NotebookLM', 'generate a Video Overview from my notebook', 'use NotebookLM Studio'). Handles reading and querying notebooks, adding sources (URLs, text, files, YouTube links, synthesized content), generating Studio outputs (Audio/Video Overviews, Mind Maps, Reports incl. Briefing Doc/Study Guide/FAQ, Flashcards, Quiz, slide decks, infographics — discover the exact set from the live Studio panel; the UI evolves fast), and creating new notebooks. Requires browser automation environment — fails gracefully when unavailable.
litreview
Academic literature orientation skill that searches papers via free keyless APIs (PubMed E-utilities + OpenAlex) by default — with the Consensus MCP as an optional enhancement lane when connected — builds a strategic search plan using PICO (default) or SPIDER / Decomposition / hybrid as fallbacks, and synthesizes findings into a formatted Word (.docx) research guide. Grill-me intake (research question specificity + framework hint + tentative depth) before the recon search; a second forcing checkpoint after Phase 2 confirms framework + sub-areas + depth before searches consume budget. Configurable depth (5/10/20 queries) controls coverage vs. speed. Output is a 'launching pad' — an orientation guide that lets a researcher dive in confidently, not a finished review. Use when the user starts literature-oriented research (e.g., 'litreview on [topic]', 'literature review on [topic]', 'I'm starting a literature review on X', 'I'm writing a paper on X', 'help me research X', 'I'm doing research on X', 'can you help me research X'). Do NOT use for single one-off paper searches wanting a quick list — that's a plain PubMed/OpenAlex (or Consensus) query.
grants
NIH grant research skill for clinical researchers. Grill-me intake (research idea + career stage + preliminary data + environment + submission posture + known institute targets) locks down the funding strategy before any search runs. Runs a 5-facet Consensus positioning analysis (with draft Significance/Innovation language), maps the research to the right NIH institutes and study sections via RePORTER, finds NOSIs and funded overlap, and produces an editable Word document (.docx) with budget/scope-aware mechanism recommendations, submission timelines, and a mandatory program officer recommendation. Use when the user asks about research funding or makes any grant-related request (e.g., 'grants for [topic]', 'find grants for my research idea', 'what grants match my research', 'help me find NIH funding', 'grant opportunities for my research'). NIH-only scope — non-NIH funders (PCORI, DOD CDMRP, VA, foundations) are out of scope and flagged at intake.