AGENT SKILL

vulnerability-csv-reporting

Generate structured CSV security audit reports from vulnerability data with proper filtering and formatting. This skill covers CSV schema design for security reports, using Python csv.DictWriter, severity-based filtering, and field mapping from JSON to tabular format.

License 许可
Apache-2.0
最近更新
2026-11-14
Stars
★ 0
Forks
1

01 前言 Front Matter

name
vulnerability-csv-reporting
description
Generate structured CSV security audit reports from vulnerability data with proper filtering and formatting. This skill covers CSV schema design for security reports, using Python csv.DictWriter, severity-based filtering, and field mapping from JSON to tabular format.

03 SKILL.md 正文

Vulnerability CSV Report Generation

This skill provides guidance on generating structured CSV reports from vulnerability scan data—a common format for security audits and compliance reporting.

Overview

CSV (Comma-Separated Values) is a widely-used format for security reports because it's:

  • Human-readable: Can be opened in Excel, Google Sheets
  • Machine-parseable: Easy to process programmatically
  • Universal: Supported by all data analysis tools
  • Lightweight: Smaller than JSON/XML formats

When to Use CSV Reports

Ideal Use Cases

  • Compliance audits requiring tabular data
  • Executive summaries for non-technical stakeholders
  • Integration with ticketing systems (Jira, ServiceNow)
  • Automated vulnerability tracking pipelines
  • Data analysis in spreadsheet tools

Limitations

  • No hierarchical data (flat structure only)
  • Limited support for nested information
  • No standard for binary data

Alternative formats: JSON (for APIs), PDF (for formal reports), HTML (for dashboards)

CSV Schema Design for Security Reports

Essential Fields

A well-designed vulnerability report CSV should include:

Field Type Description Example
Package String Vulnerable package name express
Version String Installed version 4.17.1
CVE_ID String Vulnerability identifier CVE-2022-24999
Severity Enum Risk level CRITICAL, HIGH
CVSS_Score Float/String Numeric severity score 9.8 or N/A
Fixed_Version String Patched version 4.18.0 or N/A
Title String Brief description XSS in Express.js
Url String Reference link https://nvd.nist.gov/...

Design Principles

  1. Use descriptive column names: Package not pkg, CVE_ID not id
  2. Handle missing data: Use N/A for unavailable fields, not empty strings
  3. Consistent data types: Ensure all rows have same format
  4. Include metadata: Consider adding scan date, target, tool version

Python CSV Generation with DictWriter

Why DictWriter?

Python's csv.DictWriter is ideal for structured reports:

  • Type-safe: Column names defined upfront
  • Readable: Use dictionary keys instead of indices
  • Maintainable: Easy to add/remove columns
  • Automatic header generation: No manual header writing

Basic Usage

import csv

headers = ["Package", "Version", "CVE_ID", "Severity", "CVSS_Score",
           "Fixed_Version", "Title", "Url"]

vulnerabilities = [
    {
        "Package": "minimist",
        "Version": "1.2.5",
        "CVE_ID": "CVE-2021-44906",
        "Severity": "CRITICAL",
        "CVSS_Score": 9.8,
        "Fixed_Version": "1.2.6",
        "Title": "Prototype Pollution",
        "Url": "https://avd.aquasec.com/nvd/cve-2021-44906"
    }
]

with open('security_audit.csv', 'w', newline='', encoding='utf-8') as f:
    writer = csv.DictWriter(f, fieldnames=headers)
    writer.writeheader()
    writer.writerows(vulnerabilities)

Important parameters:

  • newline='': Prevents extra blank lines on Windows
  • encoding='utf-8': Handles special characters in descriptions

Severity-Based Filtering

Why Filter by Severity?

Security teams prioritize based on risk. Filtering ensures reports focus on critical issues:

Severity Action Required Typical SLA
CRITICAL Immediate patch 24 hours
HIGH Urgent patch 7 days
MEDIUM Scheduled patch 30 days
LOW Optional patch 90 days

Implementation

def filter_high_severity(vulnerabilities, min_severity=['HIGH', 'CRITICAL']):
    """Filter vulnerabilities by severity level."""
    return [v for v in vulnerabilities if v.get('Severity') in min_severity]

Field Mapping from JSON to CSV

Extracting Fields from Scanner Output

import json

def parse_trivy_json_to_csv_records(json_file):
    """Parse Trivy JSON output and extract CSV-ready records."""
    with open(json_file, 'r', encoding='utf-8') as f:
        data = json.load(f)

    records = []

    for result in data.get('Results', []) or []:
        for vuln in result.get('Vulnerabilities', []) or []:
            record = {
                "Package": vuln.get('PkgName'),
                "Version": vuln.get('InstalledVersion'),
                "CVE_ID": vuln.get('VulnerabilityID'),
                "Severity": vuln.get('Severity', 'UNKNOWN'),
                "CVSS_Score": extract_cvss_score(vuln),
                "Fixed_Version": vuln.get('FixedVersion', 'N/A'),
                "Title": vuln.get('Title', 'No description'),
                "Url": vuln.get('PrimaryURL', ''),
            }
            records.append(record)

    return records

Canonical row selection & de-duplication (make comparisons reproducible)

When a verifier expects an exact set of rows, differences often come from accidental row explosion or from selecting a different advisory for the “same” package/version.

Make the following explicit:

  1. Row identity

    • Treat each Trivy VulnerabilityID entry as a separate row.
    • A safe unique key is (Package, Version, CVE_ID).
  2. Do not manufacture extra vulnerabilities

    • Only output vulnerabilities that appear in the scan JSON and pass the severity filter.
    • Avoid enriching the list by querying additional databases unless the task explicitly requires it.
  3. If duplicates exist in the JSON, keep a deterministic “best” row Duplicates can occur across targets/results or due to overlapping detections. If the same (Package, Version, CVE_ID) appears more than once, pick one deterministically:

    • Prefer the entry that has a non-empty PrimaryURL.
    • Then prefer one with a numeric CVSS v3 score.
    • Then prefer one with a non-empty FixedVersion.

This preserves a single canonical record per key without changing the vulnerability set.

def choose_best_duplicate(a, b):
    """Return the better of two Trivy vulnerability dicts for the same (pkg, ver, id)."""
    def score(v):
        s = 0
        if isinstance(v.get('PrimaryURL'), str) and v['PrimaryURL'].strip():
            s += 4
        cvss = v.get('CVSS') or {}
        if isinstance(cvss, dict):
            for src in ('nvd','ghsa','redhat'):
                ent = cvss.get(src)
                if isinstance(ent, dict) and isinstance(ent.get('V3Score'), (int, float)):
                    s += 2
                    break
        if isinstance(v.get('FixedVersion'), str) and v['FixedVersion'].strip():
            s += 1
        return s

    return a if score(a) >= score(b) else b

Complete Vulnerability CSV Report Generator

import json
import csv
import sys

def get_cvss_score(vuln_data):
    """Extract CVSS score with source priority (see cvss-score-extraction skill)."""
    cvss = vuln_data.get('CVSS', {})
    for source in ['nvd', 'ghsa', 'redhat']:
        if source in cvss:
            score = cvss[source].get('V3Score')
            if score is not None:
                return score
    # Optional fallback to v2 if needed:
    if 'nvd' in cvss and cvss['nvd'].get('V2Score') is not None:
        return cvss['nvd'].get('V2Score')
    return 'N/A'

def generate_vulnerability_csv_report(
    json_input,
    csv_output,
    severity_filter=['HIGH', 'CRITICAL']
):
    """Generate filtered CSV security report from Trivy JSON output."""

    try:
        with open(json_input, 'r', encoding='utf-8') as f:
            data = json.load(f)
    except FileNotFoundError:
        print(f"[!] Error: Could not find {json_input}")
        sys.exit(1)

    # Deduplicate deterministically by (pkg, version, vulnID)
    best_by_key = {}

    for result in data.get('Results', []) or []:
        for vuln in result.get('Vulnerabilities', []) or []:
            severity = vuln.get('Severity', 'UNKNOWN')
            if severity not in severity_filter:
                continue

            pkg = vuln.get('PkgName', 'Unknown')
            ver = vuln.get('InstalledVersion', 'N/A')
            vid = vuln.get('VulnerabilityID', 'N/A')
            key = (pkg, ver, vid)

            if key in best_by_key:
                # Keep a stable best record
                best_by_key[key] = choose_best_duplicate(best_by_key[key], vuln)
            else:
                best_by_key[key] = vuln

    # Map to CSV rows
    rows = []
    for (pkg, ver, vid), vuln in best_by_key.items():
        fixed = vuln.get('FixedVersion')
        if not (isinstance(fixed, str) and fixed.strip()):
            fixed = 'N/A'

        title = vuln.get('Title')
        if not (isinstance(title, str) and title.strip()):
            title = vuln.get('Description') or 'N/A'

        url = vuln.get('PrimaryURL') or ''

        rows.append({
            "Package": pkg,
            "Version": ver,
            "CVE_ID": vid,
            "Severity": vuln.get('Severity', 'UNKNOWN'),
            "CVSS_Score": get_cvss_score(vuln),
            "Fixed_Version": fixed,
            "Title": title,
            "Url": url,
        })

    headers = ["Package", "Version", "CVE_ID", "Severity",
               "CVSS_Score", "Fixed_Version", "Title", "Url"]

    # Stable ordering helps review; verifiers should be order-independent.
    rows.sort(key=lambda r: (r["Package"], r["Version"], r["CVE_ID"]))

    with open(csv_output, 'w', newline='', encoding='utf-8') as f:
        writer = csv.DictWriter(f, fieldnames=headers)
        writer.writeheader()
        writer.writerows(rows)

    print(f"[SUCCESS] Wrote {len(rows)} vulnerabilities to {csv_output}")

# Note: choose_best_duplicate is shown above.

Advanced Patterns

Pattern 1: Adding Metadata Row

import csv
from datetime import datetime

metadata = {
    "Package": f"Scan Date: {datetime.now().isoformat()}",
    "Version": "Tool: Trivy",
    "CVE_ID": "Target: package-lock.json",
    "Severity": "", "CVSS_Score": "", "Fixed_Version": "",
    "Title": "", "Url": ""
}

with open('report.csv', 'w', newline='') as f:
    writer = csv.DictWriter(f, fieldnames=headers)
    writer.writeheader()
    writer.writerow(metadata)
    writer.writerow({})
    writer.writerows(vulnerabilities)

Pattern 2: Multi-Target Reports

def generate_multi_target_report(json_input, csv_output):
    """Include target/file name in each row."""
    with open(json_input, 'r') as f:
        data = json.load(f)

    vulnerabilities = []

    for result in data.get('Results', []):
        target = result.get('Target', 'Unknown')

        for vuln in result.get('Vulnerabilities', []):
            record = {
                "Target": target,
                "Package": vuln.get('PkgName'),
            }
            vulnerabilities.append(record)

    headers = ["Target", "Package", "Version", ...]

Pattern 3: Summary Statistics

def print_report_summary(vulnerabilities):
    """Print summary before writing CSV."""
    from collections import Counter

    severity_counts = Counter(v['Severity'] for v in vulnerabilities)

    print("\nVulnerability Summary:")
    print(f"  CRITICAL: {severity_counts.get('CRITICAL', 0)}")
    print(f"  HIGH:     {severity_counts.get('HIGH', 0)}")
    print(f"  Total:    {len(vulnerabilities)}")

Error Handling

Handling Missing or Malformed Data

def safe_get_field(vuln, field, default='N/A'):
    """Safely extract field with default fallback."""
    value = vuln.get(field, default)
    return value if value is not None else default

record = {
    "Package": safe_get_field(vuln, 'PkgName', 'Unknown'),
    "Fixed_Version": safe_get_field(vuln, 'FixedVersion', 'N/A'),
}

Best Practices

  1. Always write headers: Makes CSV self-documenting
  2. Use UTF-8 encoding: Handles international characters
  3. Set newline='': Prevents blank lines on Windows
  4. Validate data: Check for None/null values before writing
  5. Filter exactly as required: If the task says only HIGH/CRITICAL, ensure no other severities are present.
  6. Do not add extra rows: Output should be a filtered projection of the scan results unless explicitly asked to enrich.
  7. Deterministic deduping: If duplicates appear in scanner JSON, select a single canonical record consistently.

Dependencies

Python Modules

  • csv (standard library)
  • json (standard library)

Input Format

  • Requires structured vulnerability data (typically JSON from scanners)

References

04 同仓库更多 Skills