[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6F66ABEXJC90W3A4JC56PJY":3,"skill-same:01M6F66ABEXJC90W3A4JC56PJY":78},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":46,"linkOnly":47,"markdownContent":48,"markdownBody":49,"frontMatter":50,"refName":55,"commitSha":56,"skillDirectory":57,"forksCount":58,"repositoryHtmlUrl":59,"repositoryHomepageUrl":6,"ownerHtmlUrl":60,"ownerBlogUrl":61,"ownerDescription":62,"updatedAt":63,"install":64,"resources":77},"01M6F66ABEXJC90W3A4JC56PJY","databricks-cost-tuning",null,"Optimize Databricks costs with cluster policies, spot instances, and monitoring.\nUse when reducing cloud spend, implementing cost controls,\nor analyzing Databricks usage costs.\nTrigger with phrases like \"databricks cost\", \"reduce databricks spend\",\n\"databricks billing\", \"databricks cost optimization\", \"cluster cost\".","Dicklesworthstone\u002Fpi_agent_rust","Dicklesworthstone","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F35050222?v=4","01M6CG26AR942CSVMN1M5E3DSE","Jeff Emanuel","01M6CG26EPP71PSHW1VDS71121",1991,"MIT","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Fplugins-community\u002Fplugins\u002Fsaas-packs\u002Fdatabricks-pack\u002Fskills\u002Fdatabricks-cost-tuning\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45],"databricks","tuning","databricks cost tuning","cost","when","cost optimization","reduce","implementing","usage","with","billing","controls","cluster","trigger","io","spot","optimize","instances","cloud spend","use","edit","write","read","Bash","analyzing","Optimization","monitoring","2026-11-10 10:12:30",false,"---\nname: databricks-cost-tuning\ndescription: |\n  Optimize Databricks costs with cluster policies, spot instances, and monitoring.\n  Use when reducing cloud spend, implementing cost controls,\n  or analyzing Databricks usage costs.\n  Trigger with phrases like \"databricks cost\", \"reduce databricks spend\",\n  \"databricks billing\", \"databricks cost optimization\", \"cluster cost\".\nallowed-tools: Read, Write, Edit, Bash(databricks:*), Grep\nversion: 1.0.0\nlicense: MIT\nauthor: Jeremy Longshore \u003Cjeremy@intentsolutions.io>\n---\n\n# Databricks Cost Tuning\n\n## Overview\nOptimize Databricks costs through cluster policies, instance selection, and monitoring.\n\n## Prerequisites\n- Workspace admin access\n- Access to billing data\n- Understanding of workload patterns\n\n## Instructions\n\n### Step 1: Implement Cluster Policies\n\n```json\n\u002F\u002F policies\u002Fcost_controlled_policy.json\n{\n  \"cluster_type\": {\n    \"type\": \"fixed\",\n    \"value\": \"job\"\n  },\n  \"autotermination_minutes\": {\n    \"type\": \"range\",\n    \"minValue\": 10,\n    \"maxValue\": 60,\n    \"defaultValue\": 30\n  },\n  \"num_workers\": {\n    \"type\": \"range\",\n    \"minValue\": 1,\n    \"maxValue\": 10,\n    \"defaultValue\": 2\n  },\n  \"node_type_id\": {\n    \"type\": \"allowlist\",\n    \"values\": [\n      \"Standard_DS3_v2\",\n      \"Standard_DS4_v2\",\n      \"Standard_E4ds_v4\"\n    ],\n    \"defaultValue\": \"Standard_DS3_v2\"\n  },\n  \"spark_version\": {\n    \"type\": \"regex\",\n    \"pattern\": \"14\\\\.[0-9]+\\\\.x-scala2\\\\.12\"\n  },\n  \"azure_attributes.spot_bid_max_price\": {\n    \"type\": \"fixed\",\n    \"value\": -1\n  },\n  \"azure_attributes.first_on_demand\": {\n    \"type\": \"fixed\",\n    \"value\": 1\n  },\n  \"custom_tags.cost_center\": {\n    \"type\": \"fixed\",\n    \"value\": \"data-engineering\"\n  }\n}\n```\n\n```python\n# Create policy via SDK\nfrom databricks.sdk import WorkspaceClient\nimport json\n\ndef create_cost_policy(w: WorkspaceClient, policy_name: str) -> str:\n    \"\"\"Create cost-controlled cluster policy.\"\"\"\n\n    policy_definition = {\n        \"autotermination_minutes\": {\n            \"type\": \"range\",\n            \"minValue\": 10,\n            \"maxValue\": 60,\n            \"defaultValue\": 30,\n        },\n        \"num_workers\": {\n            \"type\": \"range\",\n            \"minValue\": 1,\n            \"maxValue\": 10,\n        },\n        \"node_type_id\": {\n            \"type\": \"allowlist\",\n            \"values\": [\n                \"Standard_DS3_v2\",\n                \"Standard_DS4_v2\",\n            ],\n        },\n        \"azure_attributes.spot_bid_max_price\": {\n            \"type\": \"fixed\",\n            \"value\": -1,  # Use spot instances\n        },\n        \"azure_attributes.first_on_demand\": {\n            \"type\": \"fixed\",\n            \"value\": 1,  # At least 1 on-demand for driver\n        },\n    }\n\n    policy = w.cluster_policies.create(\n        name=policy_name,\n        definition=json.dumps(policy_definition),\n    )\n\n    return policy.policy_id\n```\n\n### Step 2: Spot Instance Configuration\n\n```yaml\n# resources\u002Fcost_optimized_cluster.yml\njob_clusters:\n  - job_cluster_key: cost_optimized\n    new_cluster:\n      spark_version: \"14.3.x-scala2.12\"\n      node_type_id: \"Standard_DS3_v2\"\n      num_workers: 4\n\n      # Azure Spot Configuration\n      azure_attributes:\n        first_on_demand: 1         # Driver on-demand\n        availability: SPOT_AZURE   # Workers on spot\n        spot_bid_max_price: -1     # Pay up to on-demand price\n\n      # AWS Spot Configuration (alternative)\n      # aws_attributes:\n      #   first_on_demand: 1\n      #   availability: SPOT_WITH_FALLBACK\n      #   spot_bid_price_percent: 100\n\n      autoscale:\n        min_workers: 1\n        max_workers: 8\n\n      custom_tags:\n        cost_center: analytics\n        environment: production\n```\n\n### Step 3: Instance Pool for Faster Startup\n\n```python\n# Create instance pool for reduced startup time and costs\nfrom databricks.sdk import WorkspaceClient\nfrom databricks.sdk.service.compute import (\n    InstancePoolAndStats,\n    InstancePoolAzureAttributes,\n)\n\ndef create_cost_optimized_pool(\n    w: WorkspaceClient,\n    pool_name: str,\n    min_idle: int = 1,\n    max_capacity: int = 10,\n    idle_timeout: int = 15,\n) -> str:\n    \"\"\"\n    Create instance pool for cost optimization.\n\n    Benefits:\n    - Faster cluster startup (instances pre-warmed)\n    - Lower spot instance preemption risk\n    - Consistent performance\n    \"\"\"\n    pool = w.instance_pools.create(\n        instance_pool_name=pool_name,\n        node_type_id=\"Standard_DS3_v2\",\n        min_idle_instances=min_idle,\n        max_capacity=max_capacity,\n        idle_instance_autotermination_minutes=idle_timeout,\n        azure_attributes=InstancePoolAzureAttributes(\n            availability=\"SPOT_AZURE\",\n            spot_bid_max_price=-1,\n        ),\n        preloaded_spark_versions=[\"14.3.x-scala2.12\"],\n        custom_tags={\n            \"pool_type\": \"cost_optimized\",\n            \"managed_by\": \"terraform\",\n        },\n    )\n\n    return pool.instance_pool_id\n```\n\n### Step 4: Cost Monitoring\n\n```sql\n-- Analyze cluster costs from system tables\nSELECT\n    cluster_name,\n    cluster_id,\n    DATE(usage_date) as date,\n    SUM(usage_quantity) as dbu_usage,\n    SUM(usage_quantity * list_price) as estimated_cost\nFROM system.billing.usage\nWHERE usage_date > current_date() - INTERVAL 30 DAYS\n  AND usage_type = 'COMPUTE'\nGROUP BY cluster_name, cluster_id, DATE(usage_date)\nORDER BY estimated_cost DESC;\n\n-- Top cost drivers by job\nSELECT\n    job_name,\n    COUNT(DISTINCT run_id) as runs,\n    SUM(duration) \u002F 3600000 as total_hours,\n    AVG(duration) \u002F 60000 as avg_minutes,\n    SUM(dbu_usage) as total_dbus\nFROM system.lakeflow.job_run_timeline r\nJOIN system.billing.usage u ON r.cluster_id = u.cluster_id\nWHERE r.start_time > current_timestamp() - INTERVAL 30 DAYS\nGROUP BY job_name\nORDER BY total_dbus DESC\nLIMIT 20;\n\n-- Identify idle clusters\nSELECT\n    cluster_name,\n    cluster_id,\n    state,\n    last_activity_time,\n    TIMESTAMPDIFF(HOUR, last_activity_time, current_timestamp()) as idle_hours\nFROM system.compute.clusters\nWHERE state = 'RUNNING'\n  AND TIMESTAMPDIFF(MINUTE, last_activity_time, current_timestamp()) > 30\nORDER BY idle_hours DESC;\n```\n\n### Step 5: Cost Allocation Tags\n\n```python\n# Implement cost allocation with tags\nfrom databricks.sdk import WorkspaceClient\n\ndef enforce_cost_tags(\n    w: WorkspaceClient,\n    required_tags: list[str] = [\"cost_center\", \"team\", \"environment\"],\n) -> list[dict]:\n    \"\"\"\n    Audit and report clusters missing required cost tags.\n\n    Returns list of non-compliant clusters.\n    \"\"\"\n    non_compliant = []\n\n    for cluster in w.clusters.list():\n        tags = cluster.custom_tags or {}\n        missing_tags = [t for t in required_tags if t not in tags]\n\n        if missing_tags:\n            non_compliant.append({\n                \"cluster_id\": cluster.cluster_id,\n                \"cluster_name\": cluster.cluster_name,\n                \"missing_tags\": missing_tags,\n                \"creator\": cluster.creator_user_name,\n            })\n\n    return non_compliant\n\n# Cost allocation report\ndef generate_cost_report(w: WorkspaceClient, spark) -> dict:\n    \"\"\"Generate cost allocation report by tags.\"\"\"\n    query = \"\"\"\n        SELECT\n            custom_tags:cost_center as cost_center,\n            custom_tags:team as team,\n            SUM(usage_quantity * list_price) as total_cost\n        FROM system.billing.usage\n        WHERE usage_date > current_date() - INTERVAL 30 DAYS\n        GROUP BY custom_tags:cost_center, custom_tags:team\n        ORDER BY total_cost DESC\n    \"\"\"\n    return spark.sql(query).toPandas().to_dict('records')\n```\n\n### Step 6: Auto-Termination and Scheduling\n\n```python\n# Job scheduling for cost optimization\nfrom databricks.sdk import WorkspaceClient\nfrom databricks.sdk.service.jobs import CronSchedule\n\ndef configure_off_hours_schedule(\n    w: WorkspaceClient,\n    job_id: int,\n    cron_expression: str = \"0 0 2 * * ?\",  # 2 AM daily\n    timezone: str = \"America\u002FNew_York\",\n) -> None:\n    \"\"\"Configure job to run during off-peak hours.\"\"\"\n    w.jobs.update(\n        job_id=job_id,\n        new_settings={\n            \"schedule\": {\n                \"quartz_cron_expression\": cron_expression,\n                \"timezone_id\": timezone,\n            }\n        }\n    )\n\n# Cluster auto-stop policy\ndef set_aggressive_auto_termination(\n    w: WorkspaceClient,\n    cluster_id: str,\n    minutes: int = 15,\n) -> None:\n    \"\"\"Set aggressive auto-termination for development clusters.\"\"\"\n    cluster = w.clusters.get(cluster_id)\n    w.clusters.edit(\n        cluster_id=cluster_id,\n        spark_version=cluster.spark_version,\n        node_type_id=cluster.node_type_id,\n        autotermination_minutes=minutes,\n    )\n```\n\n## Output\n- Cost-controlled cluster policies\n- Spot instance optimization\n- Instance pools configured\n- Cost monitoring dashboards\n- Auto-termination policies\n\n## Error Handling\n| Issue | Cause | Solution |\n|-------|-------|----------|\n| Spot preemption | Instance reclaimed | Use first_on_demand for stability |\n| Policy violation | Cluster over limits | Adjust policy or request exception |\n| Missing tags | Manual cluster creation | Enforce policy with required tags |\n| High idle costs | Long auto-term timeout | Reduce to 15-30 minutes |\n\n## Examples\n\n### Cost Savings Calculator\n```python\ndef estimate_spot_savings(\n    on_demand_cost: float,\n    spot_ratio: float = 0.7,  # 70% workers on spot\n    spot_discount: float = 0.6,  # 60% discount\n) -> dict:\n    \"\"\"Estimate savings from spot instances.\"\"\"\n    on_demand_portion = on_demand_cost * (1 - spot_ratio)\n    spot_portion = on_demand_cost * spot_ratio * (1 - spot_discount)\n    new_cost = on_demand_portion + spot_portion\n    savings = on_demand_cost - new_cost\n\n    return {\n        \"original_cost\": on_demand_cost,\n        \"optimized_cost\": new_cost,\n        \"savings\": savings,\n        \"savings_percent\": (savings \u002F on_demand_cost) * 100,\n    }\n```\n\n### Weekly Cost Alert\n```sql\n-- Alert when costs exceed threshold\nCREATE ALERT weekly_cost_alert\nAS SELECT\n    SUM(usage_quantity * list_price) as weekly_cost\nFROM system.billing.usage\nWHERE usage_date > current_date() - INTERVAL 7 DAYS\nHAVING weekly_cost > 10000  -- $10k threshold\nSCHEDULE CRON '0 9 * * 1'  -- Monday 9 AM\nNOTIFICATIONS (email_addresses = ['finops@company.com']);\n```\n\n## Resources\n- [Databricks Pricing](https:\u002F\u002Fdatabricks.com\u002Fproduct\u002Fpricing)\n- [Cluster Policies](https:\u002F\u002Fdocs.databricks.com\u002Fadministration-guide\u002Fclusters\u002Fpolicies.html)\n- [Instance Pools](https:\u002F\u002Fdocs.databricks.com\u002Fclusters\u002Finstance-pools\u002Findex.html)\n- [Cost Management](https:\u002F\u002Fdocs.databricks.com\u002Fadministration-guide\u002Faccount-settings\u002Fbillable-usage.html)\n\n## Next Steps\nFor reference architecture, see `databricks-reference-architecture`.\n","# Databricks Cost Tuning\n\n## Overview\nOptimize Databricks costs through cluster policies, instance selection, and monitoring.\n\n## Prerequisites\n- Workspace admin access\n- Access to billing data\n- Understanding of workload patterns\n\n## Instructions\n\n### Step 1: Implement Cluster Policies\n\n```json\n\u002F\u002F policies\u002Fcost_controlled_policy.json\n{\n  \"cluster_type\": {\n    \"type\": \"fixed\",\n    \"value\": \"job\"\n  },\n  \"autotermination_minutes\": {\n    \"type\": \"range\",\n    \"minValue\": 10,\n    \"maxValue\": 60,\n    \"defaultValue\": 30\n  },\n  \"num_workers\": {\n    \"type\": \"range\",\n    \"minValue\": 1,\n    \"maxValue\": 10,\n    \"defaultValue\": 2\n  },\n  \"node_type_id\": {\n    \"type\": \"allowlist\",\n    \"values\": [\n      \"Standard_DS3_v2\",\n      \"Standard_DS4_v2\",\n      \"Standard_E4ds_v4\"\n    ],\n    \"defaultValue\": \"Standard_DS3_v2\"\n  },\n  \"spark_version\": {\n    \"type\": \"regex\",\n    \"pattern\": \"14\\\\.[0-9]+\\\\.x-scala2\\\\.12\"\n  },\n  \"azure_attributes.spot_bid_max_price\": {\n    \"type\": \"fixed\",\n    \"value\": -1\n  },\n  \"azure_attributes.first_on_demand\": {\n    \"type\": \"fixed\",\n    \"value\": 1\n  },\n  \"custom_tags.cost_center\": {\n    \"type\": \"fixed\",\n    \"value\": \"data-engineering\"\n  }\n}\n```\n\n```python\n# Create policy via SDK\nfrom databricks.sdk import WorkspaceClient\nimport json\n\ndef create_cost_policy(w: WorkspaceClient, policy_name: str) -> str:\n    \"\"\"Create cost-controlled cluster policy.\"\"\"\n\n    policy_definition = {\n        \"autotermination_minutes\": {\n            \"type\": \"range\",\n            \"minValue\": 10,\n            \"maxValue\": 60,\n            \"defaultValue\": 30,\n        },\n        \"num_workers\": {\n            \"type\": \"range\",\n            \"minValue\": 1,\n            \"maxValue\": 10,\n        },\n        \"node_type_id\": {\n            \"type\": \"allowlist\",\n            \"values\": [\n                \"Standard_DS3_v2\",\n                \"Standard_DS4_v2\",\n            ],\n        },\n        \"azure_attributes.spot_bid_max_price\": {\n            \"type\": \"fixed\",\n            \"value\": -1,  # Use spot instances\n        },\n        \"azure_attributes.first_on_demand\": {\n            \"type\": \"fixed\",\n            \"value\": 1,  # At least 1 on-demand for driver\n        },\n    }\n\n    policy = w.cluster_policies.create(\n        name=policy_name,\n        definition=json.dumps(policy_definition),\n    )\n\n    return policy.policy_id\n```\n\n### Step 2: Spot Instance Configuration\n\n```yaml\n# resources\u002Fcost_optimized_cluster.yml\njob_clusters:\n  - job_cluster_key: cost_optimized\n    new_cluster:\n      spark_version: \"14.3.x-scala2.12\"\n      node_type_id: \"Standard_DS3_v2\"\n      num_workers: 4\n\n      # Azure Spot Configuration\n      azure_attributes:\n        first_on_demand: 1         # Driver on-demand\n        availability: SPOT_AZURE   # Workers on spot\n        spot_bid_max_price: -1     # Pay up to on-demand price\n\n      # AWS Spot Configuration (alternative)\n      # aws_attributes:\n      #   first_on_demand: 1\n      #   availability: SPOT_WITH_FALLBACK\n      #   spot_bid_price_percent: 100\n\n      autoscale:\n        min_workers: 1\n        max_workers: 8\n\n      custom_tags:\n        cost_center: analytics\n        environment: production\n```\n\n### Step 3: Instance Pool for Faster Startup\n\n```python\n# Create instance pool for reduced startup time and costs\nfrom databricks.sdk import WorkspaceClient\nfrom databricks.sdk.service.compute import (\n    InstancePoolAndStats,\n    InstancePoolAzureAttributes,\n)\n\ndef create_cost_optimized_pool(\n    w: WorkspaceClient,\n    pool_name: str,\n    min_idle: int = 1,\n    max_capacity: int = 10,\n    idle_timeout: int = 15,\n) -> str:\n    \"\"\"\n    Create instance pool for cost optimization.\n\n    Benefits:\n    - Faster cluster startup (instances pre-warmed)\n    - Lower spot instance preemption risk\n    - Consistent performance\n    \"\"\"\n    pool = w.instance_pools.create(\n        instance_pool_name=pool_name,\n        node_type_id=\"Standard_DS3_v2\",\n        min_idle_instances=min_idle,\n        max_capacity=max_capacity,\n        idle_instance_autotermination_minutes=idle_timeout,\n        azure_attributes=InstancePoolAzureAttributes(\n            availability=\"SPOT_AZURE\",\n            spot_bid_max_price=-1,\n        ),\n        preloaded_spark_versions=[\"14.3.x-scala2.12\"],\n        custom_tags={\n            \"pool_type\": \"cost_optimized\",\n            \"managed_by\": \"terraform\",\n        },\n    )\n\n    return pool.instance_pool_id\n```\n\n### Step 4: Cost Monitoring\n\n```sql\n-- Analyze cluster costs from system tables\nSELECT\n    cluster_name,\n    cluster_id,\n    DATE(usage_date) as date,\n    SUM(usage_quantity) as dbu_usage,\n    SUM(usage_quantity * list_price) as estimated_cost\nFROM system.billing.usage\nWHERE usage_date > current_date() - INTERVAL 30 DAYS\n  AND usage_type = 'COMPUTE'\nGROUP BY cluster_name, cluster_id, DATE(usage_date)\nORDER BY estimated_cost DESC;\n\n-- Top cost drivers by job\nSELECT\n    job_name,\n    COUNT(DISTINCT run_id) as runs,\n    SUM(duration) \u002F 3600000 as total_hours,\n    AVG(duration) \u002F 60000 as avg_minutes,\n    SUM(dbu_usage) as total_dbus\nFROM system.lakeflow.job_run_timeline r\nJOIN system.billing.usage u ON r.cluster_id = u.cluster_id\nWHERE r.start_time > current_timestamp() - INTERVAL 30 DAYS\nGROUP BY job_name\nORDER BY total_dbus DESC\nLIMIT 20;\n\n-- Identify idle clusters\nSELECT\n    cluster_name,\n    cluster_id,\n    state,\n    last_activity_time,\n    TIMESTAMPDIFF(HOUR, last_activity_time, current_timestamp()) as idle_hours\nFROM system.compute.clusters\nWHERE state = 'RUNNING'\n  AND TIMESTAMPDIFF(MINUTE, last_activity_time, current_timestamp()) > 30\nORDER BY idle_hours DESC;\n```\n\n### Step 5: Cost Allocation Tags\n\n```python\n# Implement cost allocation with tags\nfrom databricks.sdk import WorkspaceClient\n\ndef enforce_cost_tags(\n    w: WorkspaceClient,\n    required_tags: list[str] = [\"cost_center\", \"team\", \"environment\"],\n) -> list[dict]:\n    \"\"\"\n    Audit and report clusters missing required cost tags.\n\n    Returns list of non-compliant clusters.\n    \"\"\"\n    non_compliant = []\n\n    for cluster in w.clusters.list():\n        tags = cluster.custom_tags or {}\n        missing_tags = [t for t in required_tags if t not in tags]\n\n        if missing_tags:\n            non_compliant.append({\n                \"cluster_id\": cluster.cluster_id,\n                \"cluster_name\": cluster.cluster_name,\n                \"missing_tags\": missing_tags,\n                \"creator\": cluster.creator_user_name,\n            })\n\n    return non_compliant\n\n# Cost allocation report\ndef generate_cost_report(w: WorkspaceClient, spark) -> dict:\n    \"\"\"Generate cost allocation report by tags.\"\"\"\n    query = \"\"\"\n        SELECT\n            custom_tags:cost_center as cost_center,\n            custom_tags:team as team,\n            SUM(usage_quantity * list_price) as total_cost\n        FROM system.billing.usage\n        WHERE usage_date > current_date() - INTERVAL 30 DAYS\n        GROUP BY custom_tags:cost_center, custom_tags:team\n        ORDER BY total_cost DESC\n    \"\"\"\n    return spark.sql(query).toPandas().to_dict('records')\n```\n\n### Step 6: Auto-Termination and Scheduling\n\n```python\n# Job scheduling for cost optimization\nfrom databricks.sdk import WorkspaceClient\nfrom databricks.sdk.service.jobs import CronSchedule\n\ndef configure_off_hours_schedule(\n    w: WorkspaceClient,\n    job_id: int,\n    cron_expression: str = \"0 0 2 * * ?\",  # 2 AM daily\n    timezone: str = \"America\u002FNew_York\",\n) -> None:\n    \"\"\"Configure job to run during off-peak hours.\"\"\"\n    w.jobs.update(\n        job_id=job_id,\n        new_settings={\n            \"schedule\": {\n                \"quartz_cron_expression\": cron_expression,\n                \"timezone_id\": timezone,\n            }\n        }\n    )\n\n# Cluster auto-stop policy\ndef set_aggressive_auto_termination(\n    w: WorkspaceClient,\n    cluster_id: str,\n    minutes: int = 15,\n) -> None:\n    \"\"\"Set aggressive auto-termination for development clusters.\"\"\"\n    cluster = w.clusters.get(cluster_id)\n    w.clusters.edit(\n        cluster_id=cluster_id,\n        spark_version=cluster.spark_version,\n        node_type_id=cluster.node_type_id,\n        autotermination_minutes=minutes,\n    )\n```\n\n## Output\n- Cost-controlled cluster policies\n- Spot instance optimization\n- Instance pools configured\n- Cost monitoring dashboards\n- Auto-termination policies\n\n## Error Handling\n| Issue | Cause | Solution |\n|-------|-------|----------|\n| Spot preemption | Instance reclaimed | Use first_on_demand for stability |\n| Policy violation | Cluster over limits | Adjust policy or request exception |\n| Missing tags | Manual cluster creation | Enforce policy with required tags |\n| High idle costs | Long auto-term timeout | Reduce to 15-30 minutes |\n\n## Examples\n\n### Cost Savings Calculator\n```python\ndef estimate_spot_savings(\n    on_demand_cost: float,\n    spot_ratio: float = 0.7,  # 70% workers on spot\n    spot_discount: float = 0.6,  # 60% discount\n) -> dict:\n    \"\"\"Estimate savings from spot instances.\"\"\"\n    on_demand_portion = on_demand_cost * (1 - spot_ratio)\n    spot_portion = on_demand_cost * spot_ratio * (1 - spot_discount)\n    new_cost = on_demand_portion + spot_portion\n    savings = on_demand_cost - new_cost\n\n    return {\n        \"original_cost\": on_demand_cost,\n        \"optimized_cost\": new_cost,\n        \"savings\": savings,\n        \"savings_percent\": (savings \u002F on_demand_cost) * 100,\n    }\n```\n\n### Weekly Cost Alert\n```sql\n-- Alert when costs exceed threshold\nCREATE ALERT weekly_cost_alert\nAS SELECT\n    SUM(usage_quantity * list_price) as weekly_cost\nFROM system.billing.usage\nWHERE usage_date > current_date() - INTERVAL 7 DAYS\nHAVING weekly_cost > 10000  -- $10k threshold\nSCHEDULE CRON '0 9 * * 1'  -- Monday 9 AM\nNOTIFICATIONS (email_addresses = ['finops@company.com']);\n```\n\n## Resources\n- [Databricks Pricing](https:\u002F\u002Fdatabricks.com\u002Fproduct\u002Fpricing)\n- [Cluster Policies](https:\u002F\u002Fdocs.databricks.com\u002Fadministration-guide\u002Fclusters\u002Fpolicies.html)\n- [Instance Pools](https:\u002F\u002Fdocs.databricks.com\u002Fclusters\u002Finstance-pools\u002Findex.html)\n- [Cost Management](https:\u002F\u002Fdocs.databricks.com\u002Fadministration-guide\u002Faccount-settings\u002Fbillable-usage.html)\n\n## Next Steps\nFor reference architecture, see `databricks-reference-architecture`.\n",{"name":5,"author":51,"license":15,"version":52,"description":53,"allowed-tools":54},"Jeremy Longshore \u003Cjeremy@intentsolutions.io>","1.0.0","Optimize Databricks costs with cluster policies, spot instances, and monitoring.\nUse when reducing cloud spend, implementing cost controls,\nor analyzing Databricks usage costs.\nTrigger with phrases like \"databricks cost\", \"reduce databricks spend\",\n\"databricks billing\", \"databricks cost optimization\", \"cluster cost\".\n","Read, Write, Edit, Bash(databricks:*), Grep","main","90b50ed0778745a80fd5bfb13b1843d45b9e82f7","tests\u002Fext_conformance\u002Fartifacts\u002Fplugins-community\u002Fplugins\u002Fsaas-packs\u002Fdatabricks-pack\u002Fskills\u002Fdatabricks-cost-tuning",234,"https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust","https:\u002F\u002Fgithub.com\u002FDicklesworthstone","https:\u002F\u002Fwww.jeffreyemanuel.com\u002F","Building in NY","2026-11-13 11:50:27",{"source":8,"skillName":5,"commands":65,"prompt":75,"zipAvailable":76},[66,69,72],{"tool":67,"command":68},"npx","npx skills add Dicklesworthstone\u002Fpi_agent_rust --skill databricks-cost-tuning",{"tool":70,"command":71},"bunx","bunx skills add Dicklesworthstone\u002Fpi_agent_rust --skill databricks-cost-tuning",{"tool":73,"command":74},"pnpm","pnpm dlx skills add Dicklesworthstone\u002Fpi_agent_rust --skill databricks-cost-tuning","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Fplugins-community\u002Fplugins\u002Fsaas-packs\u002Fdatabricks-pack\u002Fskills\u002Fdatabricks-cost-tuning\u002FSKILL.md",true,[],[79,119,143,169,196,218],{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":83,"licenseFileUrl":84,"sourceHtmlUrl":85,"categories":86,"keywords":92,"publishedAt":46},"01M6FEGPPK0ZTKCCM0R3E3KS33","web-artifacts-builder","Suite of tools for creating elaborate, multi-component claude.ai HTML artifacts using modern frontend web technologies (React, Tailwind CSS, shadcn\u002Fui). Use for complex artifacts requiring state management, routing, or shadcn\u002Fui components - not for simple single-file HTML\u002FJSX artifacts.","Apache-2.0","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Futilities\u002Fweb-artifacts-builder\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Futilities\u002Fweb-artifacts-builder\u002FSKILL.md",[87],{"categoryNo":88,"categoryName":89,"taxonomyType":90,"taxonomyTypeDesc":91},"01JBSEEDCAT000000000000009","前端开发","SCENARIO","使用场景",[93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,38,111,112,113,114,115,116,117,118],"web artifacts builder","web","artifacts","artifacts builder","builder","frontend","css","components","react","component","routing","state management","state","shadcn","file","ui","Tools","Tailwind","html","single-file","simple","ui components","multi","for","jsx","suite",{"skillNo":120,"name":121,"nameZh":6,"description":122,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":83,"licenseFileUrl":123,"sourceHtmlUrl":124,"categories":125,"keywords":130,"publishedAt":46},"01M6FEE9VTK1W4GY21AKREJXRG","webapp-testing","Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fsecurity\u002Fwebapp-testing\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fsecurity\u002Fwebapp-testing\u002FSKILL.md",[126,129],{"categoryNo":127,"categoryName":128,"taxonomyType":90,"taxonomyTypeDesc":91},"01JBSEEDCAT000000000000008","测试与质量",{"categoryNo":88,"categoryName":89,"taxonomyType":90,"taxonomyTypeDesc":91},[131,132,133,98,134,135,94,136,137,138,108,139,140,141,142,28,116],"testing","webapp testing","webapp","screenshots","playwright","debugging","toolkit","browser","logs","applications","behavior","local",{"skillNo":144,"name":145,"nameZh":6,"description":146,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":147,"sourceHtmlUrl":148,"categories":149,"keywords":150,"publishedAt":46},"01M6FE20F7ZTQBFWKF4MPJ2SBX","neuropixels-analysis","Neuropixels neural recording analysis. Load SpikeGLX\u002FOpenEphys data, preprocess, motion correction, Kilosort4 spike sorting, quality metrics, Allen\u002FIBL curation, AI-assisted visual analysis, for Neuropixels 1.0\u002F2.0 extracellular electrophysiology. Use when working with neural recordings, spike sorting, extracellular electrophysiology, or when the user mentions Neuropixels, SpikeGLX, Open Ephys, Kilosort, quality metrics, or unit curation.","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fscientific\u002Fneuropixels-analysis\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fscientific\u002Fneuropixels-analysis\u002FSKILL.md",[],[151,152,153,154,155,156,157,158,38,159,160,161,162,163,164,165,28,166,167,168,116,23],"analysis","neuropixels analysis","spike","metrics","data","open","quality","user","recordings","Neural","motion","visual","quality metrics","unit","sorting","curation","recording","load",{"skillNo":170,"name":171,"nameZh":6,"description":172,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":83,"licenseFileUrl":173,"sourceHtmlUrl":174,"categories":175,"keywords":179,"publishedAt":46},"01M6FDP0PDK6F1GGX395C7625Z","skill-creator","Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Claude's capabilities with specialized knowledge, workflows, or tool integrations.","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fproductivity\u002Fskill-creator\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fproductivity\u002Fskill-creator\u002FSKILL.md",[176],{"categoryNo":177,"categoryName":178,"taxonomyType":90,"taxonomyTypeDesc":91},"01JBSEEDCAT000000000000023","Skill与智能体开发",[180,181,182,183,184,185,186,187,188,189,190,191,192,193,28,116,194,23,195],"skill creator","creator","workflows","create","update","capabilities","knowledge","tool","integrations","specialized","guide","new skill","existing skill","users","Existing","new",{"skillNo":197,"name":198,"nameZh":6,"description":199,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":200,"sourceHtmlUrl":201,"categories":202,"keywords":204,"publishedAt":46},"01M6FDM8Y7VK7NYG9SM7QPA3PW","notebooklm","Use this skill to query your Google NotebookLM notebooks directly from Claude Code for source-grounded, citation-backed answers from Gemini. Browser automation, library management, persistent auth. Drastically reduced hallucinations through document-only responses.","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fproductivity\u002Fnotebooklm\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fproductivity\u002Fnotebooklm\u002FSKILL.md",[203],{"categoryNo":177,"categoryName":178,"taxonomyType":90,"taxonomyTypeDesc":91},[198,205,206,207,208,209,210,138,211,212,38,213,214,215,216,116,217],"google","browser automation","gemini","automation","auth","code","library","management","only","document","query","Citation","source",{"skillNo":219,"name":220,"nameZh":6,"description":221,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":83,"licenseFileUrl":222,"sourceHtmlUrl":223,"categories":224,"keywords":231,"publishedAt":46},"01M6FDHPXE10J3XGTCH5HBHS1M","slack-gif-creator","Knowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like \"make me a GIF of X doing Y for Slack.\"","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fenterprise-communication\u002Fslack-gif-creator\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fenterprise-communication\u002Fslack-gif-creator\u002FSKILL.md",[225,228],{"categoryNo":226,"categoryName":227,"taxonomyType":90,"taxonomyTypeDesc":91},"01JBSEEDCAT000000000000005","邮件与沟通",{"categoryNo":229,"categoryName":230,"taxonomyType":90,"taxonomyTypeDesc":91},"01JBSEEDCAT000000000000016","图像与多媒体",[232,233,234,181,235,109,236,237,186,38,238,193,239,240,241,116,242,23],"slack gif creator","slack","gif","validation","constraints","Animation","me","make","Utilities","concepts","request"]