[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M78CHHD9VA7Z5G49GTY76Z2D":3,"skill-same:01M78CHHD9VA7Z5G49GTY76Z2D":71},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":28,"commitSha":29,"skillDirectory":30,"forksCount":14,"repositoryHtmlUrl":31,"repositoryHomepageUrl":32,"ownerHtmlUrl":33,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":34,"install":35,"resources":48},"01M78CHHD9VA7Z5G49GTY76Z2D","gcp-spark",null,"Develops and executes Spark code on Dataproc Clusters and Serverless.\nReads and writes data using BigLake Iceberg catalogs, BigQuery and Spanner.\nDebugs execution failures.\nUse when:\n- Writing Spark ETL pipelines on GCP.\n- Training or running inference with ML models with spark on GCP.\n- Managing Spark clusters, jobs, batches, and interactive sessions.\nDon't use when:\n- Writing generic Python scripts that don't use Spark.\n- Performing simple SQL queries that can be done directly in BigQuery.","princepatel04477-web\u002Ftrivoxa-group","princepatel04477-web","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F227740714?v=4","01M788ZAJ2KECD5KXVP55PZXKS","Prince Patel","01M788ZAJTCN8YSD737MRSKC5Q",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web\u002Ftrivoxa-group\u002Fblob\u002Fmaster\u002F.github\u002Fskills\u002Fgcp-spark\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: gcp-spark\ndescription: |\n  Develops and executes Spark code on Dataproc Clusters and Serverless.\n  Reads and writes data using BigLake Iceberg catalogs, BigQuery and Spanner.\n  Debugs execution failures.\n  Use when:\n  - Writing Spark ETL pipelines on GCP.\n  - Training or running inference with ML models with spark on GCP.\n  - Managing Spark clusters, jobs, batches, and interactive sessions.\n  Don't use when:\n  - Writing generic Python scripts that don't use Spark.\n  - Performing simple SQL queries that can be done directly in BigQuery.\nlicense: Apache-2.0\nmetadata:\n  version: v2\n  publisher: google\n---\n\n# Spark on Dataproc\n\n> [!IMPORTANT] You MUST ALWAYS follow the Task Execution Workflow when writing\n> spark code.\n\n## Task Execution Workflow\n\n1.  **Understand schemas**: **ALWAYS** use `@skill:discovering-gcp-data-assets`\n    skill or `resources\u002Fschema_direct_inspection.md` to understand input and\n    output schemas. Include the schema in your thought process BEFORE generating\n    any code. Do NOT guess column names.\n2.  **Generate spark code**:\n    *   **Output Format**: **ALWAYS** generate code in **Python Notebooks\n        (.ipynb)** format. Generate scripts (.py) only if explicitly requested.\n    *   **Read and Write data**: **ALWAYS** Refer to\n        `resources\u002Fread_write_data.md` when reading or writing data.\n    *   **ML Tasks**: Refer to `@skill:ml-best-practices` skill and\n        `resources\u002Fml_tasks.md` when generating ML code.\n    *   **Spark Optimizations**: **ALWAYS** refer to\n        `resources\u002Fspark_optimizations.md` when generating spark code and apply\n        optimization whenever applicable.\n3.  **Verify schema before write**: **ALWAYS** verify that the dataframe and\n    destination schema match, use `df.printSchema()` for dataframe schema and\n    refer to `@skill:discovering-gcp-data-assets` skill or\n    `resources\u002Fschema_direct_inspection.md` to verify destination schema.\n4.  **Compile code before executing**: For notebooks convert them to python\n    script using `jupyter nbconvert --to script your-notebook.ipynb` first, then\n    compile code using `python3 -m py_compile your-notebook.py`.\n5.  **Execute script**: ONLY when generating a `.py` script refer to\n    `resources\u002Fgcloud_dataproc.md` on writing command to execute generated code\n    on Dataproc. This DOES NOT apply when generating notebooks.\n\n--------------------------------------------------------------------------------\n\n## Common Mistakes Checklist\n\n> [!CAUTION] Ensure you verify this checklist to avoid mistakes\n\nBefore submitting a job, verify:\n\n-   [ ] **All imports present** (`col`, `when`, `lit`, etc. from\n    `pyspark.sql.functions`)\n-   [ ] **`vector_to_array` from correct module** use `from pyspark.ml.functions\n    import vector_to_array` (NOT `pyspark.sql.functions`)\n-   [ ] **DataFrame schema matches target Iceberg table** verify with\n    `df.printSchema()` before writing\n-   [ ] **CSV files read with `header` and `inferSchema`** without these, the\n    header row becomes data and all columns are strings\n-   [ ] **Avoid toPandas()** Converting a pyspark dataframe to pandas by calling\n    toPandas() can lead to out of memory errors. Only acceptable for building\n    visualizations in Spark 3.5\n\n--------------------------------------------------------------------------------\n\n## IAM Requirements\n\nThe Dataproc service account needs:\n\n*   `roles\u002Fdataproc.worker`: Job execution\n*   `roles\u002Fbiglake.admin`: Iceberg table management\n*   `roles\u002Fbigquery.jobUser`: Query materialization\n*   `roles\u002Fstorage.objectUser`: Read\u002Fwrite GCS\n*   `roles\u002Fspanner.databaseUser`: Spanner writes\n\n--------------------------------------------------------------------------------\n\n## Spark resource management\n\nRefer to `resources\u002Fgcloud_dataproc.md` for detailed guidelines on managing\nSpark clusters, jobs, batches, and interactive sessions.\n","# Spark on Dataproc\n\n> [!IMPORTANT] You MUST ALWAYS follow the Task Execution Workflow when writing\n> spark code.\n\n## Task Execution Workflow\n\n1.  **Understand schemas**: **ALWAYS** use `@skill:discovering-gcp-data-assets`\n    skill or `resources\u002Fschema_direct_inspection.md` to understand input and\n    output schemas. Include the schema in your thought process BEFORE generating\n    any code. Do NOT guess column names.\n2.  **Generate spark code**:\n    *   **Output Format**: **ALWAYS** generate code in **Python Notebooks\n        (.ipynb)** format. Generate scripts (.py) only if explicitly requested.\n    *   **Read and Write data**: **ALWAYS** Refer to\n        `resources\u002Fread_write_data.md` when reading or writing data.\n    *   **ML Tasks**: Refer to `@skill:ml-best-practices` skill and\n        `resources\u002Fml_tasks.md` when generating ML code.\n    *   **Spark Optimizations**: **ALWAYS** refer to\n        `resources\u002Fspark_optimizations.md` when generating spark code and apply\n        optimization whenever applicable.\n3.  **Verify schema before write**: **ALWAYS** verify that the dataframe and\n    destination schema match, use `df.printSchema()` for dataframe schema and\n    refer to `@skill:discovering-gcp-data-assets` skill or\n    `resources\u002Fschema_direct_inspection.md` to verify destination schema.\n4.  **Compile code before executing**: For notebooks convert them to python\n    script using `jupyter nbconvert --to script your-notebook.ipynb` first, then\n    compile code using `python3 -m py_compile your-notebook.py`.\n5.  **Execute script**: ONLY when generating a `.py` script refer to\n    `resources\u002Fgcloud_dataproc.md` on writing command to execute generated code\n    on Dataproc. This DOES NOT apply when generating notebooks.\n\n--------------------------------------------------------------------------------\n\n## Common Mistakes Checklist\n\n> [!CAUTION] Ensure you verify this checklist to avoid mistakes\n\nBefore submitting a job, verify:\n\n-   [ ] **All imports present** (`col`, `when`, `lit`, etc. from\n    `pyspark.sql.functions`)\n-   [ ] **`vector_to_array` from correct module** use `from pyspark.ml.functions\n    import vector_to_array` (NOT `pyspark.sql.functions`)\n-   [ ] **DataFrame schema matches target Iceberg table** verify with\n    `df.printSchema()` before writing\n-   [ ] **CSV files read with `header` and `inferSchema`** without these, the\n    header row becomes data and all columns are strings\n-   [ ] **Avoid toPandas()** Converting a pyspark dataframe to pandas by calling\n    toPandas() can lead to out of memory errors. Only acceptable for building\n    visualizations in Spark 3.5\n\n--------------------------------------------------------------------------------\n\n## IAM Requirements\n\nThe Dataproc service account needs:\n\n*   `roles\u002Fdataproc.worker`: Job execution\n*   `roles\u002Fbiglake.admin`: Iceberg table management\n*   `roles\u002Fbigquery.jobUser`: Query materialization\n*   `roles\u002Fstorage.objectUser`: Read\u002Fwrite GCS\n*   `roles\u002Fspanner.databaseUser`: Spanner writes\n\n--------------------------------------------------------------------------------\n\n## Spark resource management\n\nRefer to `resources\u002Fgcloud_dataproc.md` for detailed guidelines on managing\nSpark clusters, jobs, batches, and interactive sessions.\n",{"name":5,"license":15,"metadata":24,"description":27},{"version":25,"publisher":26},"v2","google","Develops and executes Spark code on Dataproc Clusters and Serverless.\nReads and writes data using BigLake Iceberg catalogs, BigQuery and Spanner.\nDebugs execution failures.\nUse when:\n- Writing Spark ETL pipelines on GCP.\n- Training or running inference with ML models with spark on GCP.\n- Managing Spark clusters, jobs, batches, and interactive sessions.\nDon't use when:\n- Writing generic Python scripts that don't use Spark.\n- Performing simple SQL queries that can be done directly in BigQuery.\n","master","877653a671c51fae6476bd9f0d34ee6c408d68b9",".github\u002Fskills\u002Fgcp-spark","https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web\u002Ftrivoxa-group","https:\u002F\u002Ftrivoxa-group.vercel.app","https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web","2026-08-01 17:29:31",{"source":8,"skillName":5,"commands":36,"prompt":46,"zipAvailable":47},[37,40,43],{"tool":38,"command":39},"npx","npx skills add princepatel04477-web\u002Ftrivoxa-group --skill gcp-spark",{"tool":41,"command":42},"bunx","bunx skills add princepatel04477-web\u002Ftrivoxa-group --skill gcp-spark",{"tool":44,"command":45},"pnpm","pnpm dlx skills add princepatel04477-web\u002Ftrivoxa-group --skill gcp-spark","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web\u002Ftrivoxa-group\u002Fblob\u002Fmaster\u002F.github\u002Fskills\u002Fgcp-spark\u002FSKILL.md",true,[49,55,59,63,67],{"path":50,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":53,"sourceHtmlUrl":54},"resources\u002Fgcloud_dataproc.md","REFERENCE","参考文档",7819,"https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web\u002Ftrivoxa-group\u002Fblob\u002Fmaster\u002F.github\u002Fskills\u002Fgcp-spark\u002Fresources\u002Fgcloud_dataproc.md",{"path":56,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":57,"sourceHtmlUrl":58},"resources\u002Fml_tasks.md",1895,"https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web\u002Ftrivoxa-group\u002Fblob\u002Fmaster\u002F.github\u002Fskills\u002Fgcp-spark\u002Fresources\u002Fml_tasks.md",{"path":60,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":61,"sourceHtmlUrl":62},"resources\u002Fread_write_data.md",13575,"https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web\u002Ftrivoxa-group\u002Fblob\u002Fmaster\u002F.github\u002Fskills\u002Fgcp-spark\u002Fresources\u002Fread_write_data.md",{"path":64,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":65,"sourceHtmlUrl":66},"resources\u002Fschema_direct_inspection.md",595,"https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web\u002Ftrivoxa-group\u002Fblob\u002Fmaster\u002F.github\u002Fskills\u002Fgcp-spark\u002Fresources\u002Fschema_direct_inspection.md",{"path":68,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":69,"sourceHtmlUrl":70},"resources\u002Fspark_optimizations.md",322,"https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web\u002Ftrivoxa-group\u002Fblob\u002Fmaster\u002F.github\u002Fskills\u002Fgcp-spark\u002Fresources\u002Fspark_optimizations.md",[72,80,87,94,101,108],{"skillNo":73,"name":74,"nameZh":6,"description":75,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":76,"licenseFileUrl":6,"sourceHtmlUrl":77,"categories":78,"keywords":79,"publishedAt":19},"01M78G4Z5RFJ72G536XYPC77PG","zustand-store-ts","Create Zustand stores following established patterns with proper TypeScript types and middleware.","MIT","https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web\u002Ftrivoxa-group\u002Fblob\u002Fmaster\u002F.github\u002Fskills\u002Fzustand-store-ts\u002FSKILL.md",[],[],{"skillNo":81,"name":82,"nameZh":6,"description":83,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":76,"licenseFileUrl":6,"sourceHtmlUrl":84,"categories":85,"keywords":86,"publishedAt":19},"01M78G4WQMHRST36KDP7AV3NR3","zoom-automation","Automate Zoom meeting creation, management, recordings, webinars, and participant tracking via Rube MCP (Composio). Always search tools first for current schemas.","https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web\u002Ftrivoxa-group\u002Fblob\u002Fmaster\u002F.github\u002Fskills\u002Fzoom-automation\u002FSKILL.md",[],[],{"skillNo":88,"name":89,"nameZh":6,"description":90,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":76,"licenseFileUrl":6,"sourceHtmlUrl":91,"categories":92,"keywords":93,"publishedAt":19},"01M78G4T37F4SPS7M4KT0A8JJB","zoho-crm-automation","Automate Zoho CRM tasks via Rube MCP (Composio): create\u002Fupdate records, search contacts, manage leads, and convert leads. Always search tools first for current schemas.","https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web\u002Ftrivoxa-group\u002Fblob\u002Fmaster\u002F.github\u002Fskills\u002Fzoho-crm-automation\u002FSKILL.md",[],[],{"skillNo":95,"name":96,"nameZh":6,"description":97,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":76,"licenseFileUrl":6,"sourceHtmlUrl":98,"categories":99,"keywords":100,"publishedAt":19},"01M78G4QAJMF71KD1G38MNDQYK","zod-validation-expert","Expert in Zod — TypeScript-first schema validation. Covers parsing, custom errors, refinements, type inference, and integration with React Hook Form, Next.js, and tRPC.","https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web\u002Ftrivoxa-group\u002Fblob\u002Fmaster\u002F.github\u002Fskills\u002Fzod-validation-expert\u002FSKILL.md",[],[],{"skillNo":102,"name":103,"nameZh":6,"description":104,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":76,"licenseFileUrl":6,"sourceHtmlUrl":105,"categories":106,"keywords":107,"publishedAt":19},"01M78G4MS3WE96MH5P561KFY95","zipai-optimizer","Adaptive token optimizer: intelligent filtering, surgical output, ambiguity-first, context-window-aware, VCS-aware, MCP-aware.","https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web\u002Ftrivoxa-group\u002Fblob\u002Fmaster\u002F.github\u002Fskills\u002Fzipai-optimizer\u002FSKILL.md",[],[],{"skillNo":109,"name":110,"nameZh":6,"description":111,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":76,"licenseFileUrl":6,"sourceHtmlUrl":112,"categories":113,"keywords":114,"publishedAt":19},"01M78G4GZ0T47C6929V9CVH1AK","zeroize-audit","Detects missing zeroization of sensitive data in source code and identifies zeroization removed by compiler optimizations, with assembly-level analysis, and control-flow verification. Use for auditing C\u002FC++\u002FRust code handling secrets, keys, passwords, or other sensitive data.","https:\u002F\u002Fgithub.com\u002Fprincepatel04477-web\u002Ftrivoxa-group\u002Fblob\u002Fmaster\u002F.github\u002Fskills\u002Fzeroize-audit\u002FSKILL.md",[],[]]