implementing-aws-macie-for-data-classification
Enable and configure Amazon Macie via AWS CLI/Terraform to discover, classify, and protect sensitive data (PII, financial data, credentials) in S3 using ML and pattern matching, including discovery jobs, custom data identifiers, allow lists, and EventBridge-based remediation. Use when setting up S3 data classification, cloud DLP, or auditing S3 for unprotected sensitive data.
- License 许可
- Apache-2.0
- 最近更新
- 2026-08-25
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- implementing-aws-macie-for-data-classification
- tags
- [ "aws", "macie", "data-classification", "s3", "pii", "sensitive-data", "dlp", "compliance" ]
- author
- mahipal
- domain
- cybersecurity
- license
- Apache-2.0
- version
- 1.0
- nist_csf
- [ "PR.IR-01", "ID.AM-08", "GV.SC-06", "DE.CM-01" ]
- subdomain
- cloud-security
- description
- Enable and configure Amazon Macie via AWS CLI/Terraform to discover, classify, and protect sensitive data (PII, financial data, credentials) in S3 using ML and pattern matching, including discovery jobs, custom data identifiers, allow lists, and EventBridge-based remediation. Use when setting up S3 data classification, cloud DLP, or auditing S3 for unprotected sensitive data.
- nist_ai_rmf
- [ "GOVERN-1.1", "GOVERN-4.2", "MAP-2.3", "MEASURE-2.7", "MEASURE-2.5" ]
- mitre_attack
- [ "T1078.004", "T1530", "T1537", "T1580", "T1003" ]
- atlas_techniques
- [ "AML.T0043", "AML.T0018" ]
02 资源文件 7
- 资产文件assets/template.md775 B
- 许可证LICENSE11.0 KB
- 参考文档references/api-reference.md2.0 KB
- 参考文档references/standards.md625 B
- 参考文档references/workflows.md823 B
- 脚本scripts/agent.py6.0 KB
- 脚本scripts/process.py5.5 KB
03 SKILL.md 正文
Implementing AWS Macie for Data Classification
Overview
Amazon Macie is a fully managed data security and privacy service that uses machine learning and pattern matching to discover and protect sensitive data in Amazon S3. Macie automatically evaluates your S3 bucket inventory on a daily basis and identifies objects containing PII, financial information, credentials, and other sensitive data types. It provides two discovery approaches: automated sensitive data discovery for broad visibility and targeted discovery jobs for deep analysis.
When to Use
- When deploying or configuring implementing aws macie for data classification capabilities in your environment
- When establishing security controls aligned to compliance requirements
- When building or improving security architecture for this domain
- When conducting security assessments that require this implementation
Prerequisites
- AWS account with S3 buckets containing data to classify
- IAM permissions for Macie service configuration
- AWS Organizations setup (for multi-account deployment)
- S3 buckets in supported regions
Enable Macie
Via AWS CLI
# Enable Macie in the current account/region
aws macie2 enable-macie
# Verify Macie is enabled
aws macie2 get-macie-session
# Enable automated sensitive data discovery
aws macie2 update-automated-discovery-configuration \
--status ENABLED
Via Terraform
resource "aws_macie2_account" "main" {}
resource "aws_macie2_classification_export_configuration" "main" {
depends_on = [aws_macie2_account.main]
s3_destination {
bucket_name = aws_s3_bucket.macie_results.id
key_prefix = "macie-findings/"
kms_key_arn = aws_kms_key.macie.arn
}
}
Configure Discovery Jobs
Create a classification job for specific buckets
aws macie2 create-classification-job \
--job-type ONE_TIME \
--name "pii-scan-production-buckets" \
--s3-job-definition '{
"bucketDefinitions": [{
"accountId": "123456789012",
"buckets": [
"production-data-bucket",
"customer-records-bucket"
]
}]
}' \
--managed-data-identifier-selector ALL
Create a scheduled recurring job
aws macie2 create-classification-job \
--job-type SCHEDULED \
--name "weekly-sensitive-data-scan" \
--schedule-frequency-details '{
"weekly": {
"dayOfWeek": "MONDAY"
}
}' \
--s3-job-definition '{
"bucketDefinitions": [{
"accountId": "123456789012",
"buckets": ["all-data-bucket"]
}],
"scoping": {
"includes": {
"and": [{
"simpleScopeTerm": {
"comparator": "STARTS_WITH",
"key": "OBJECT_KEY",
"values": ["uploads/", "documents/"]
}
}]
}
}
}'
Custom Data Identifiers
Create a custom identifier for internal IDs
aws macie2 create-custom-data-identifier \
--name "internal-employee-id" \
--description "Matches internal employee ID format EMP-XXXXXX" \
--regex "EMP-[0-9]{6}" \
--severity-levels '[
{"occurrencesThreshold": 1, "severity": "LOW"},
{"occurrencesThreshold": 10, "severity": "MEDIUM"},
{"occurrencesThreshold": 50, "severity": "HIGH"}
]'
Create identifier for project codes
aws macie2 create-custom-data-identifier \
--name "project-code-identifier" \
--description "Matches project codes in format PRJ-XXXX-XX" \
--regex "PRJ-[A-Z]{4}-[0-9]{2}" \
--keywords '["project", "code", "initiative"]' \
--maximum-match-distance 50
Allow Lists
Create an allow list to suppress false positives
aws macie2 create-allow-list \
--name "test-data-exclusions" \
--description "Exclude known test data patterns" \
--criteria '{
"regex": "TEST-[0-9]{4}-[0-9]{4}-[0-9]{4}-[0-9]{4}"
}'
Managed Data Identifiers
Macie provides 300+ managed data identifiers covering:
| Category | Examples |
|---|---|
| PII | SSN, passport numbers, driver's license, date of birth, names, addresses |
| Financial | Credit card numbers, bank account numbers, SWIFT codes |
| Credentials | AWS secret keys, API keys, SSH private keys, OAuth tokens |
| Health | HIPAA identifiers, health insurance claim numbers |
| Legal | Tax identification numbers, national ID numbers |
Findings Management
List findings
# Get sensitive data findings
aws macie2 list-findings \
--finding-criteria '{
"criterion": {
"severity.description": {
"eq": ["High"]
},
"category": {
"eq": ["CLASSIFICATION"]
}
}
}' \
--sort-criteria '{"attributeName": "updatedAt", "orderBy": "DESC"}' \
--max-results 25
Get finding details
aws macie2 get-findings \
--finding-ids '["finding-id-1", "finding-id-2"]'
Export findings to Security Hub
# Macie automatically publishes findings to Security Hub
# Verify integration:
aws macie2 get-macie-session --query 'findingPublishingFrequency'
EventBridge Integration for Automated Response
{
"source": ["aws.macie"],
"detail-type": ["Macie Finding"],
"detail": {
"severity": {
"description": ["High", "Critical"]
}
}
}
Lambda function for automated remediation
import boto3
import json
s3 = boto3.client('s3')
sns = boto3.client('sns')
def lambda_handler(event, context):
finding = event['detail']
severity = finding['severity']['description']
bucket = finding['resourcesAffected']['s3Bucket']['name']
key = finding['resourcesAffected']['s3Object']['key']
sensitive_types = [d['type'] for d in finding.get('classificationDetails', {}).get('result', {}).get('sensitiveData', [])]
if severity in ['High', 'Critical']:
# Tag the object for review
s3.put_object_tagging(
Bucket=bucket,
Key=key,
Tagging={
'TagSet': [
{'Key': 'macie-finding', 'Value': severity},
{'Key': 'sensitive-data', 'Value': ','.join(sensitive_types)},
{'Key': 'requires-review', 'Value': 'true'}
]
}
)
# Notify security team
sns.publish(
TopicArn='arn:aws:sns:us-east-1:123456789012:security-alerts',
Subject=f'Macie {severity} Finding: {bucket}/{key}',
Message=json.dumps({
'bucket': bucket,
'key': key,
'severity': severity,
'sensitive_data_types': sensitive_types,
'finding_id': finding['id']
}, indent=2)
)
return {'statusCode': 200}
Multi-Account Deployment
Designate Macie administrator account
# From the management account
aws macie2 enable-organization-admin-account \
--admin-account-id 111111111111
Add member accounts
# From the administrator account
aws macie2 create-member \
--account '{"accountId": "222222222222", "email": "security@example.com"}'
Monitoring Macie Operations
Usage statistics
aws macie2 get-usage-statistics \
--filter-by '[{"comparator": "GT", "key": "accountId", "values": []}]' \
--sort-by '{"key": "accountId", "orderBy": "ASC"}'
Classification job status
aws macie2 list-classification-jobs \
--filter-criteria '{"includes": [{"comparator": "EQ", "key": "jobStatus", "values": ["RUNNING"]}]}'
References
- AWS Macie Documentation: https://docs.aws.amazon.com/macie/
- AWS Macie Pricing
- Supported File Types for Macie Analysis
- GDPR and CCPA Compliance with Macie
04 同仓库更多 Skills
syllabus
Generates a curated supplementary reading list from any course syllabus using Consensus academic search. Grill-me intake (syllabus input format + course audience + year range) plus a grouping forcing-options checkpoint before any search runs — so the reading list matches the course's level and recency need. Parses the syllabus to extract topics and learning outcomes, searches Consensus for recent peer-reviewed papers per topic, and produces a professionally formatted .docx with clickable Consensus links, plain-language summaries calibrated to audience level, and Bloom-higher-order discussion questions tied to course learning goals. Use when the user uploads a syllabus, course outline, or curriculum document and wants supplementary readings (e.g., 'create a reading list from this syllabus', 'find recent papers for my course') — even casual mentions with a syllabus attached should trigger this skill.
pulse
Multi-source recency research skill that takes the pulse of any topic across Reddit, Hacker News, the open web, and optionally X/Twitter within a configurable recent window (default 30 days). Forcing intake clarifies topic specificity, angle (trend/sentiment/problems/opportunities/comparison), time window, and platform scope before searching. Returns a synthesized briefing with citations, engagement metrics, and cross-platform pattern analysis. Use when the user requests multi-source recency intelligence on a topic (e.g., 'pulse on [topic]', 'what's happening with [topic]', 'what are people saying about [topic]', 'current conversation about [topic]', 'take the pulse of [topic]', 'trending: [topic]', 'find me info on [topic]'), and for competitor research, trend discovery, tool comparisons, and audience sentiment analysis.
patent
Patent prior-art and landscape intelligence skill — not generic patent help. Commits to one of five sub-use-cases via forcing intake (novelty search / freedom-to-operate / competitive landscape / acquisition diligence / litigation prior-art) before any search runs. Searches Google Patents, Espacenet, USPTO, and optionally Lens.org for citation-graph signals. Output is an editable Word document (.docx) with verdict, ranked closest art (claim-text extracted), CPC-class-aware landscape, family-resolved hits, geographic coverage, FTO flags where applicable, strategy recommendations, and full audit log. Use when the user asks for patent searching or analysis (e.g., 'prior art search for [invention]', 'freedom to operate analysis for [product]'). Produces search signal, not legal advice — always recommends consulting a patent attorney before filing or licensing decisions. Trademark, copyright, and trade-secret questions are out of scope.
notebooklm
Browser automation skill for controlling Google's NotebookLM. Use when the user wants anything done in NotebookLM (e.g., 'open NotebookLM', 'check my [name] notebook', 'ask my notebook about X', 'add [source] to NotebookLM', 'generate a Video Overview from my notebook', 'use NotebookLM Studio'). Handles reading and querying notebooks, adding sources (URLs, text, files, YouTube links, synthesized content), generating Studio outputs (Audio/Video Overviews, Mind Maps, Reports incl. Briefing Doc/Study Guide/FAQ, Flashcards, Quiz, slide decks, infographics — discover the exact set from the live Studio panel; the UI evolves fast), and creating new notebooks. Requires browser automation environment — fails gracefully when unavailable.
litreview
Academic literature orientation skill that searches papers via free keyless APIs (PubMed E-utilities + OpenAlex) by default — with the Consensus MCP as an optional enhancement lane when connected — builds a strategic search plan using PICO (default) or SPIDER / Decomposition / hybrid as fallbacks, and synthesizes findings into a formatted Word (.docx) research guide. Grill-me intake (research question specificity + framework hint + tentative depth) before the recon search; a second forcing checkpoint after Phase 2 confirms framework + sub-areas + depth before searches consume budget. Configurable depth (5/10/20 queries) controls coverage vs. speed. Output is a 'launching pad' — an orientation guide that lets a researcher dive in confidently, not a finished review. Use when the user starts literature-oriented research (e.g., 'litreview on [topic]', 'literature review on [topic]', 'I'm starting a literature review on X', 'I'm writing a paper on X', 'help me research X', 'I'm doing research on X', 'can you help me research X'). Do NOT use for single one-off paper searches wanting a quick list — that's a plain PubMed/OpenAlex (or Consensus) query.
grants
NIH grant research skill for clinical researchers. Grill-me intake (research idea + career stage + preliminary data + environment + submission posture + known institute targets) locks down the funding strategy before any search runs. Runs a 5-facet Consensus positioning analysis (with draft Significance/Innovation language), maps the research to the right NIH institutes and study sections via RePORTER, finds NOSIs and funded overlap, and produces an editable Word document (.docx) with budget/scope-aware mechanism recommendations, submission timelines, and a mandatory program officer recommendation. Use when the user asks about research funding or makes any grant-related request (e.g., 'grants for [topic]', 'find grants for my research idea', 'what grants match my research', 'help me find NIH funding', 'grant opportunities for my research'). NIH-only scope — non-NIH funders (PCORI, DOD CDMRP, VA, foundations) are out of scope and flagged at intake.