databricks-security-basics
Apply Databricks security best practices for secrets and access control. Use when securing API tokens, implementing least privilege access, or auditing Databricks security configuration. Trigger with phrases like "databricks security", "databricks secrets", "secure databricks", "databricks token security", "databricks scopes".
- License 许可
- MIT
- 最近更新
- 2026-05-26
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- databricks-security-basics
- author
- Jeremy Longshore <jeremy@intentsolutions.io>
- license
- MIT
- version
- 1.0.0
- description
- Apply Databricks security best practices for secrets and access control. Use when securing API tokens, implementing least privilege access, or auditing Databricks security configuration. Trigger with phrases like "databricks security", "databricks secrets", "secure databricks", "databricks token security", "databricks scopes".
- allowed-tools
- Read, Write, Grep, Bash(databricks:*)
- compatible-with
- claude-code, codex, openclaw
03 SKILL.md 正文
Databricks Security Basics
Overview
Security best practices for Databricks tokens, secrets, and access control.
Prerequisites
- Databricks CLI installed
- Workspace admin access (for secrets management)
- Understanding of Unity Catalog
Instructions
Step 1: Configure Secret Scopes
# Create a secret scope (Databricks-backed)
databricks secrets create-scope my-app-secrets
# Create a secret scope (Azure Key Vault-backed)
databricks secrets create-scope azure-secrets \
--scope-backend-type AZURE_KEYVAULT \
--resource-id "/subscriptions/.../vaults/my-vault" \
--dns-name "https://my-vault.vault.azure.net/"
# List scopes
databricks secrets list-scopes
Step 2: Store Secrets
# Store a secret
databricks secrets put-secret my-app-secrets db-password
# Store from file (for multi-line secrets)
databricks secrets put-secret my-app-secrets api-key --string-value "sk_live_..."
# List secrets in scope (values are hidden)
databricks secrets list-secrets my-app-secrets
Step 3: Access Secrets in Code
# In notebooks and jobs
db_password = dbutils.secrets.get(scope="my-app-secrets", key="db-password")
api_key = dbutils.secrets.get(scope="my-app-secrets", key="api-key")
# Secrets are redacted in logs
print(f"Password: {db_password}") # Shows [REDACTED]
# Use in connection strings
jdbc_url = f"jdbc:postgresql://host:5432/db?user=app&password={db_password}" # 5432: PostgreSQL port
Step 4: Secret Scope ACLs
# Grant read access to a user
databricks secrets put-acl my-app-secrets user@company.com READ
# Grant manage access to a group
databricks secrets put-acl my-app-secrets data-engineers MANAGE
# List ACLs
databricks secrets list-acls my-app-secrets
Step 5: Token Management
# src/databricks/tokens.py
from databricks.sdk import WorkspaceClient
from datetime import datetime, timedelta
def audit_tokens(w: WorkspaceClient) -> list[dict]:
"""Audit all tokens in workspace for security review."""
tokens = list(w.tokens.list())
token_audit = []
for token in tokens:
created = datetime.fromtimestamp(token.creation_time / 1000) # 1000: 1 second in ms
expiry = datetime.fromtimestamp(token.expiry_time / 1000) if token.expiry_time else None # 1 second in ms
token_audit.append({
"token_id": token.token_id,
"comment": token.comment,
"created": created,
"expires": expiry,
"days_until_expiry": (expiry - datetime.now()).days if expiry else None,
"is_expired": expiry < datetime.now() if expiry else False,
"needs_rotation": expiry and (expiry - datetime.now()).days < 30 if expiry else True,
})
return token_audit
def rotate_token(w: WorkspaceClient, old_token_id: str, lifetime_days: int = 90) -> str:
"""Rotate a token by creating new and deleting old."""
# Create new token
new_token = w.tokens.create(
comment=f"Rotated token - {datetime.now().isoformat()}",
lifetime_seconds=lifetime_days * 24 * 60 * 60,
)
# Delete old token
w.tokens.delete(token_id=old_token_id)
return new_token.token_value
Step 6: Implement Least Privilege
# Environment-specific permissions
permission_matrix = {
"development": {
"cluster_access": "CAN_ATTACH_TO",
"job_access": "CAN_MANAGE_RUN",
"notebook_access": "CAN_EDIT",
},
"staging": {
"cluster_access": "CAN_RESTART",
"job_access": "CAN_MANAGE_RUN",
"notebook_access": "CAN_READ",
},
"production": {
"cluster_access": "CAN_ATTACH_TO", # Read-only
"job_access": "CAN_VIEW",
"notebook_access": "CAN_READ",
},
}
def apply_environment_permissions(
w: WorkspaceClient,
environment: str,
user_email: str,
resource_type: str,
resource_id: str,
) -> None:
"""Apply environment-appropriate permissions."""
perms = permission_matrix[environment]
if resource_type == "cluster":
w.permissions.update(
object_type="clusters",
object_id=resource_id,
access_control_list=[{
"user_name": user_email,
"permission_level": perms["cluster_access"],
}]
)
Output
- Secret scopes configured
- Secrets stored securely
- Token rotation procedures
- Least privilege access
Error Handling
| Security Issue | Detection | Mitigation |
|---|---|---|
| Exposed token | Audit logs | Immediate rotation |
| Excessive scopes | Permission audit | Reduce to minimum |
| No token expiry | Token audit | Set 90-day max lifetime |
| Shared credentials | Usage patterns | Individual service principals |
Examples
Security Audit Script
from databricks.sdk import WorkspaceClient
def security_audit(w: WorkspaceClient) -> dict:
"""Run comprehensive security audit."""
audit_results = {
"tokens": [],
"secrets": [],
"permissions": [],
"findings": [],
}
# Audit tokens
for token in w.tokens.list():
if not token.expiry_time:
audit_results["findings"].append({
"severity": "HIGH",
"type": "token_no_expiry",
"detail": f"Token {token.comment} has no expiration",
})
# Audit secret scopes
for scope in w.secrets.list_scopes():
acls = list(w.secrets.list_acls(scope=scope.name))
if not acls:
audit_results["findings"].append({
"severity": "MEDIUM",
"type": "scope_no_acl",
"detail": f"Scope {scope.name} has no ACLs",
})
return audit_results
Service Principal Setup (Azure)
# Create service principal in Azure AD
az ad sp create-for-rbac --name databricks-cicd
# Add to Databricks workspace (Admin Console or SCIM API)
databricks service-principals create \
--json '{
"display_name": "CI/CD Pipeline",
"application_id": "your-azure-app-id"
}'
# Grant permissions
databricks permissions update jobs --job-id 123 --json '{
"access_control_list": [{
"service_principal_name": "your-azure-app-id",
"permission_level": "CAN_MANAGE"
}]
}'
Unity Catalog Security
-- Create secure group
CREATE GROUP IF NOT EXISTS `data-scientists`;
-- Grant schema access (not table-level for simplicity)
GRANT USAGE ON CATALOG main TO `data-scientists`;
GRANT USAGE ON SCHEMA main.analytics TO `data-scientists`;
GRANT SELECT ON SCHEMA main.analytics TO `data-scientists`;
-- Audit grants
SHOW GRANTS ON SCHEMA main.analytics;
-- Row-level security
ALTER TABLE main.analytics.customers
SET ROW FILTER customer_region_filter ON (region);
Security Checklist
- [ ] Personal access tokens have expiration dates
- [ ] Secrets stored in Databricks Secret Scopes
- [ ] No hardcoded credentials in notebooks
- [ ] Service principals for automation
- [ ] Unity Catalog for data governance
- [ ] Audit logging enabled
- [ ] Network security (IP access lists)
- [ ] Cluster policies enforced
Resources
Next Steps
For production deployment, see databricks-prod-checklist.
04 同仓库更多 Skills
zustand-store-ts
Create Zustand stores with TypeScript, subscribeWithSelector middleware, and proper state/action separation. Use when building React state management, creating global stores, or implementing reactive state patterns with Zustand.
zustand-store-creator
Zustand Store Creator - Auto-activating skill for Frontend Development. Triggers on: zustand store creator, zustand store creator Part of the Frontend Development skill category.
zero-trust-config-helper
Zero Trust Config Helper - Auto-activating skill for Security Advanced. Triggers on: zero trust config helper, zero trust config helper Part of the Security Advanced skill category.
zapier-integration-helper
Zapier Integration Helper - Auto-activating skill for Business Automation. Triggers on: zapier integration helper, zapier integration helper Part of the Business Automation skill category.
zai-cli
Execute z.AI CLI providing vision, search, reader, and GitHub exploration via CLI and MCP. Use when user needs image/video analysis, OCR, UI-to-code conversion, error diagnosis, real-time web search, web page to markdown extraction, or GitHub code exploration. Trigger with phrases like "analyze this image", "search the web for", "read this page", "explore this repo", or "use zai". Requires Z_AI_API_KEY.
yt-research
Research competitor YouTube channels, niches, and trending topics for your content strategy. Use this skill whenever the user says "research channels", "analyze competitors", "find trending topics", "niche analysis", "competitive research", "what are other creators doing", "scrape YouTube channels", or wants to understand the competitive landscape for a specific tool or topic area. Use when working with yt research. Trigger with 'yt', 'research'.