[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M77KYE864ESSHCK0T4YCD5Z8":3,"skill-same:01M77KYE864ESSHCK0T4YCD5Z8":46},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":27,"repositoryHtmlUrl":28,"repositoryHomepageUrl":6,"ownerHtmlUrl":29,"ownerBlogUrl":30,"ownerDescription":6,"updatedAt":31,"install":32,"resources":45},"01M77KYE864ESSHCK0T4YCD5Z8","bigquery-pipeline-audit",null,"Audits Python + BigQuery pipelines for cost safety, idempotency, and production readiness. Returns a structured report with exact patch locations.","Reihantt6\u002Fsns-myagent","Reihantt6","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F219822095?v=4","01M77KK94PV8B7C2D9XWXNS8DJ","Mohamad Reihan tegar prasetya","01M77KK958QFMTHYXGXFV7KZ79",2,"MIT","https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fbigquery-pipeline-audit\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: bigquery-pipeline-audit\ndescription: 'Audits Python + BigQuery pipelines for cost safety, idempotency, and production readiness. Returns a structured report with exact patch locations.'\n---\n\n# BigQuery Pipeline Audit: Cost, Safety and Production Readiness\n\nYou are a senior data engineer reviewing a Python + BigQuery pipeline script.\nYour goals: catch runaway costs before they happen, ensure reruns do not corrupt\ndata, and make sure failures are visible.\n\nAnalyze the codebase and respond in the structure below (A to F + Final).\nReference exact function names and line locations. Suggest minimal fixes, not\nrewrites.\n\n---\n\n## A) COST EXPOSURE: What will actually get billed?\n\nLocate every BigQuery job trigger (`client.query`, `load_table_from_*`,\n`extract_table`, `copy_table`, DDL\u002FDML via query) and every external call\n(APIs, LLM calls, storage writes).\n\nFor each, answer:\n- Is this inside a loop, retry block, or async gather?\n- What is the realistic worst-case call count?\n- For each `client.query`, is `QueryJobConfig.maximum_bytes_billed` set?\n  For load, extract, and copy jobs, is the scope bounded and counted against MAX_JOBS?\n- Is the same SQL and params being executed more than once in a single run?\n  Flag repeated identical queries and suggest query hashing plus temp table caching.\n\n**Flag immediately if:**\n- Any BQ query runs once per date or once per entity in a loop\n- Worst-case BQ job count exceeds 20\n- `maximum_bytes_billed` is missing on any `client.query` call\n\n---\n\n## B) DRY RUN AND EXECUTION MODES\n\nVerify a `--mode` flag exists with at least `dry_run` and `execute` options.\n\n- `dry_run` must print the plan and estimated scope with zero billed BQ execution\n  (BigQuery dry-run estimation via job config is allowed) and zero external API or LLM calls\n- `execute` requires explicit confirmation for prod (`--env=prod --confirm`)\n- Prod must not be the default environment\n\nIf missing, propose a minimal `argparse` patch with safe defaults.\n\n---\n\n## C) BACKFILL AND LOOP DESIGN\n\n**Hard fail if:** the script runs one BQ query per date or per entity in a loop.\n\nCheck that date-range backfills use one of:\n1. A single set-based query with `GENERATE_DATE_ARRAY`\n2. A staging table loaded with all dates then one join query\n3. Explicit chunks with a hard `MAX_CHUNKS` cap\n\nAlso check:\n- Is the date range bounded by default (suggest 14 days max without `--override`)?\n- If the script crashes mid-run, is it safe to re-run without double-writing?\n- For backdated simulations, verify data is read from time-consistent snapshots\n  (`FOR SYSTEM_TIME AS OF`, partitioned as-of tables, or dated snapshot tables).\n  Flag any read from a \"latest\" or unversioned table when running in backdated mode.\n\nSuggest a concrete rewrite if the current approach is row-by-row.\n\n---\n\n## D) QUERY SAFETY AND SCAN SIZE\n\nFor each query, check:\n- **Partition filter** is on the raw column, not `DATE(ts)`, `CAST(...)`, or\n  any function that prevents pruning\n- **No `SELECT *`**: only columns actually used downstream\n- **Joins will not explode**: verify join keys are unique or appropriately scoped\n  and flag any potential many-to-many\n- **Expensive operations** (`REGEXP`, `JSON_EXTRACT`, UDFs) only run after\n  partition filtering, not on full table scans\n\nProvide a specific SQL fix for any query that fails these checks.\n\n---\n\n## E) SAFE WRITES AND IDEMPOTENCY\n\nIdentify every write operation. Flag plain `INSERT`\u002Fappend with no dedup logic.\n\nEach write should use one of:\n1. `MERGE` on a deterministic key (e.g., `entity_id + date + model_version`)\n2. Write to a staging table scoped to the run, then swap or merge into final\n3. Append-only with a dedupe view:\n   `QUALIFY ROW_NUMBER() OVER (PARTITION BY \u003Ckey>) = 1`\n\nAlso check:\n- Will a re-run create duplicate rows?\n- Is the write disposition (`WRITE_TRUNCATE` vs `WRITE_APPEND`) intentional\n  and documented?\n- Is `run_id` being used as part of the merge or dedupe key? If so, flag it.\n  `run_id` should be stored as a metadata column, not as part of the uniqueness\n  key, unless you explicitly want multi-run history.\n\nState the recommended approach and the exact dedup key for this codebase.\n\n---\n\n## F) OBSERVABILITY: Can you debug a failure?\n\nVerify:\n- Failures raise exceptions and abort with no silent `except: pass` or warn-only\n- Each BQ job logs: job ID, bytes processed or billed when available,\n  slot milliseconds, and duration\n- A run summary is logged or written at the end containing:\n  `run_id, env, mode, date_range, tables written, total BQ jobs, total bytes`\n- `run_id` is present and consistent across all log lines\n\nIf `run_id` is missing, propose a one-line fix:\n`run_id = run_id or datetime.utcnow().strftime('%Y%m%dT%H%M%S')`\n\n---\n\n## Final\n\n**1. PASS \u002F FAIL** with specific reasons per section (A to F).\n**2. Patch list** ordered by risk, referencing exact functions to change.\n**3. If FAIL: Top 3 cost risks** with a rough worst-case estimate\n(e.g., \"loop over 90 dates x 3 retries = 270 BQ jobs\").\n","# BigQuery Pipeline Audit: Cost, Safety and Production Readiness\n\nYou are a senior data engineer reviewing a Python + BigQuery pipeline script.\nYour goals: catch runaway costs before they happen, ensure reruns do not corrupt\ndata, and make sure failures are visible.\n\nAnalyze the codebase and respond in the structure below (A to F + Final).\nReference exact function names and line locations. Suggest minimal fixes, not\nrewrites.\n\n---\n\n## A) COST EXPOSURE: What will actually get billed?\n\nLocate every BigQuery job trigger (`client.query`, `load_table_from_*`,\n`extract_table`, `copy_table`, DDL\u002FDML via query) and every external call\n(APIs, LLM calls, storage writes).\n\nFor each, answer:\n- Is this inside a loop, retry block, or async gather?\n- What is the realistic worst-case call count?\n- For each `client.query`, is `QueryJobConfig.maximum_bytes_billed` set?\n  For load, extract, and copy jobs, is the scope bounded and counted against MAX_JOBS?\n- Is the same SQL and params being executed more than once in a single run?\n  Flag repeated identical queries and suggest query hashing plus temp table caching.\n\n**Flag immediately if:**\n- Any BQ query runs once per date or once per entity in a loop\n- Worst-case BQ job count exceeds 20\n- `maximum_bytes_billed` is missing on any `client.query` call\n\n---\n\n## B) DRY RUN AND EXECUTION MODES\n\nVerify a `--mode` flag exists with at least `dry_run` and `execute` options.\n\n- `dry_run` must print the plan and estimated scope with zero billed BQ execution\n  (BigQuery dry-run estimation via job config is allowed) and zero external API or LLM calls\n- `execute` requires explicit confirmation for prod (`--env=prod --confirm`)\n- Prod must not be the default environment\n\nIf missing, propose a minimal `argparse` patch with safe defaults.\n\n---\n\n## C) BACKFILL AND LOOP DESIGN\n\n**Hard fail if:** the script runs one BQ query per date or per entity in a loop.\n\nCheck that date-range backfills use one of:\n1. A single set-based query with `GENERATE_DATE_ARRAY`\n2. A staging table loaded with all dates then one join query\n3. Explicit chunks with a hard `MAX_CHUNKS` cap\n\nAlso check:\n- Is the date range bounded by default (suggest 14 days max without `--override`)?\n- If the script crashes mid-run, is it safe to re-run without double-writing?\n- For backdated simulations, verify data is read from time-consistent snapshots\n  (`FOR SYSTEM_TIME AS OF`, partitioned as-of tables, or dated snapshot tables).\n  Flag any read from a \"latest\" or unversioned table when running in backdated mode.\n\nSuggest a concrete rewrite if the current approach is row-by-row.\n\n---\n\n## D) QUERY SAFETY AND SCAN SIZE\n\nFor each query, check:\n- **Partition filter** is on the raw column, not `DATE(ts)`, `CAST(...)`, or\n  any function that prevents pruning\n- **No `SELECT *`**: only columns actually used downstream\n- **Joins will not explode**: verify join keys are unique or appropriately scoped\n  and flag any potential many-to-many\n- **Expensive operations** (`REGEXP`, `JSON_EXTRACT`, UDFs) only run after\n  partition filtering, not on full table scans\n\nProvide a specific SQL fix for any query that fails these checks.\n\n---\n\n## E) SAFE WRITES AND IDEMPOTENCY\n\nIdentify every write operation. Flag plain `INSERT`\u002Fappend with no dedup logic.\n\nEach write should use one of:\n1. `MERGE` on a deterministic key (e.g., `entity_id + date + model_version`)\n2. Write to a staging table scoped to the run, then swap or merge into final\n3. Append-only with a dedupe view:\n   `QUALIFY ROW_NUMBER() OVER (PARTITION BY \u003Ckey>) = 1`\n\nAlso check:\n- Will a re-run create duplicate rows?\n- Is the write disposition (`WRITE_TRUNCATE` vs `WRITE_APPEND`) intentional\n  and documented?\n- Is `run_id` being used as part of the merge or dedupe key? If so, flag it.\n  `run_id` should be stored as a metadata column, not as part of the uniqueness\n  key, unless you explicitly want multi-run history.\n\nState the recommended approach and the exact dedup key for this codebase.\n\n---\n\n## F) OBSERVABILITY: Can you debug a failure?\n\nVerify:\n- Failures raise exceptions and abort with no silent `except: pass` or warn-only\n- Each BQ job logs: job ID, bytes processed or billed when available,\n  slot milliseconds, and duration\n- A run summary is logged or written at the end containing:\n  `run_id, env, mode, date_range, tables written, total BQ jobs, total bytes`\n- `run_id` is present and consistent across all log lines\n\nIf `run_id` is missing, propose a one-line fix:\n`run_id = run_id or datetime.utcnow().strftime('%Y%m%dT%H%M%S')`\n\n---\n\n## Final\n\n**1. PASS \u002F FAIL** with specific reasons per section (A to F).\n**2. Patch list** ordered by risk, referencing exact functions to change.\n**3. If FAIL: Top 3 cost risks** with a rough worst-case estimate\n(e.g., \"loop over 90 dates x 3 retries = 270 BQ jobs\").\n",{"name":5,"description":7},"main","58843b159d7a4050d6b785a98b908cce505c0b92",".agents\u002Fskills\u002Fbigquery-pipeline-audit",0,"https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent","https:\u002F\u002Fgithub.com\u002FReihantt6","reihsns.web.id","2026-08-19 20:08:18",{"source":8,"skillName":5,"commands":33,"prompt":43,"zipAvailable":44},[34,37,40],{"tool":35,"command":36},"npx","npx skills add Reihantt6\u002Fsns-myagent --skill bigquery-pipeline-audit",{"tool":38,"command":39},"bunx","bunx skills add Reihantt6\u002Fsns-myagent --skill bigquery-pipeline-audit",{"tool":41,"command":42},"pnpm","pnpm dlx skills add Reihantt6\u002Fsns-myagent --skill bigquery-pipeline-audit","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fbigquery-pipeline-audit\u002FSKILL.md",true,[],[47,54,61,68,75,82],{"skillNo":48,"name":49,"nameZh":6,"description":50,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":51,"categories":52,"keywords":53,"publishedAt":19},"01M77XDCYZ407XYMF2T4P6F7VT","youtube-transcript","Extract transcripts from YouTube videos. Use when the user asks for a transcript, subtitles, or captions of a YouTube video and provides a YouTube URL (youtube.com\u002Fwatch?v=, youtu.be\u002F, or similar). Supports output with or without timestamps.","https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002Fagent\u002Fskills\u002Fyoutube-transcript\u002FSKILL.md",[],[],{"skillNo":55,"name":56,"nameZh":6,"description":57,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":58,"categories":59,"keywords":60,"publishedAt":19},"01M77XD7PBC98J6MXGJJ2TJ8DX","youtube-thumbnail-design","YouTube thumbnail design with specific dimensions, contrast rules, and mobile preview optimization. Covers safe zones, text placement, face expression psychology, and A\u002FB testing. Use for: YouTube thumbnails, video cover images, click-through optimization. Triggers: youtube thumbnail, thumbnail design, video thumbnail, click through rate, ctr optimization, youtube cover, video cover image, thumbnail maker, thumbnail tips, youtube design, video preview image","https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002Fagent\u002Fskills\u002Fyoutube-thumbnail-design\u002FSKILL.md",[],[],{"skillNo":62,"name":63,"nameZh":6,"description":64,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":65,"categories":66,"keywords":67,"publishedAt":19},"01M77XD2541C8HXJ6RHYZP4NGJ","youtube-short-maker","Make YouTube Shorts that hold attention with Pexo — vertical 9:16, fast-paced, hook in the first 3 seconds. Describe your topic and Pexo writes the script, generates the shots, picks the models, and assembles a finished Short with music and captions. Use for YouTube Shorts: \"youtube short\", \"make a youtube short\", \"shorts video\", \"short-form video\". NOT for long-form or landscape video.","https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002Fagent\u002Fskills\u002Fyoutube-short-maker\u002FSKILL.md",[],[],{"skillNo":69,"name":70,"nameZh":6,"description":71,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":72,"categories":73,"keywords":74,"publishedAt":19},"01M77XCXKVSYK7KTZ3PQBVVGFD","xcode-project-setup","Safely modifies Xcode projects (.pbxproj) to add Swift Packages and link files. Use this skill whenever an iOS project needs dependencies installed (e.g. Firebase, Alamofire).","https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002Fagent\u002Fskills\u002Fxcode-project-setup\u002FSKILL.md",[],[],{"skillNo":76,"name":77,"nameZh":6,"description":78,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":79,"categories":80,"keywords":81,"publishedAt":19},"01M77XCR1DV8ZQK2NHWKVY4EWX","x402","Set up Browser Use Cloud payments with x402 — pay per request from a crypto wallet (USDC on Base mainnet), no signup or API key. Two setups it works out up front — \"just use it\" (set up a wallet so you or Claude Code can run cloud browser tasks paid from the wallet — Claude writes and runs throwaway scripts, nothing touches your codebase) or \"build it in\" (install the SDK and write the key + code into your project). Walks through wallet setup, funding, .env, and a ~$1 test run. Use when the user asks about x402, pay-per-use, USDC payments, or wants Browser Use Cloud without an API key. For the free-tier signup (reverse-CAPTCHA → API key), use `browser-use cloud signup` or the `cloud` skill instead.","https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002Fagent\u002Fskills\u002Fx402\u002FSKILL.md",[],[],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":86,"categories":87,"keywords":88,"publishedAt":19},"01M77XCMD3R3ZBZQQSW6E0793B","x-twitter-scraper","Build GitHub Copilot workflows with Xquik X API SDKs, REST endpoints, hosted Apify Actor runs, MCP tools, TweetClaw OpenClaw plugin installs, signed webhooks, tweet search, user lookup, follower exports, media actions, and agent automation.","https:\u002F\u002Fgithub.com\u002FReihantt6\u002Fsns-myagent\u002Fblob\u002Fmain\u002Fagent\u002Fskills\u002Fx-twitter-scraper\u002FSKILL.md",[],[]]