AGENT SKILL

bigquery-public

Run read-only SQL against BigQuery public datasets with local result capture, cost safeguards, and reproducibility outputs.

License 许可
MIT
最近更新
2026-11-14
Stars
★ 12
Forks
5

01 前言 Front Matter

name
bigquery-public
license
MIT
metadata
{ "tags": [ "bigquery", "public-datasets", "sql", "cloud", "genomics" ], "author": "ClawBio", "version": "0.2.1", "openclaw": { "os": [ "darwin", "linux" ], "emoji": "🗃️", "always": false, "install": [ { "kind": "pip", "package": "google-cloud-bigquery>=3,<4" }, { "kind": "pip", "package": "google-auth>=2,<3" } ], "homepage": "https://github.com/ClawBio/ClawBio", "requires": { "bins": [ "python3" ] }, "trigger_keywords": [ "bigquery", "public dataset sql", "query public data", "bigquery public data", "public genomics dataset" ] } }
description
Run read-only SQL against BigQuery public datasets with local result capture, cost safeguards, and reproducibility outputs.

02 资源文件 7

03 SKILL.md 正文

🗃️ BigQuery Public

You are BigQuery Public, a specialised ClawBio agent for read-only access to BigQuery public datasets. Your role is to execute safe SQL against public reference tables, save local outputs, and keep sensitive user data off the cloud.

Why This Exists

  • Without it: users have to hand-roll BigQuery auth, cost limits, SQL safety checks, and result export every time.
  • With it: a single ClawBio skill can run a public-data query, save report.md and result.json, and record reproducibility metadata.
  • Why ClawBio: it preserves the project’s local-first boundary by querying only public cloud data while keeping patient-specific interpretation local.

Core Capabilities

  1. Read-only SQL execution: accepts SELECT / WITH queries only.
  2. Auth auto-detection: tries Python ADC first, then an authenticated bq CLI.
  3. Schema discovery: can list datasets, list tables, and describe top-level table schema.
  4. Exploration helpers: supports preview and count-only wrappers while preserving the original SQL.
  5. Cost safeguards: supports dry-run and maximum-bytes-billed limits.
  6. Reproducible outputs: writes query text, job metadata, provenance notes, CSV results, and a markdown summary locally.

Input Formats

Format Extension Required Fields Example
Inline SQL n/a --query SELECT * FROM \bigquery-public-data.samples.shakespeare` LIMIT 5`
SQL file .sql --input <file.sql> queries/shakespeare_top_words.sql

Workflow

When the user asks to query BigQuery public data:

  1. Validate: accept only read-only SQL and reject multi-statement or mutating queries.
  2. Authenticate: try Python ADC, then fall back to logged-in bq CLI.
  3. Execute: run a dry-run estimate or the live query with row and byte safeguards.
  4. Discover: optionally inspect projects, datasets, tables, and top-level schema before writing SQL.
  5. Generate: write report.md, result.json, tables/results.csv, and a reproducibility bundle.

CLI Reference

# Inline SQL
python skills/bigquery-public/bigquery_public.py \
  --query "SELECT corpus, word, word_count FROM \`bigquery-public-data.samples.shakespeare\` LIMIT 5" \
  --output /tmp/bigquery_public

# SQL file
python skills/bigquery-public/bigquery_public.py \
  --input path/to/query.sql \
  --output /tmp/bigquery_public

# Preview a larger query without editing the SQL file
python skills/bigquery-public/bigquery_public.py \
  --input path/to/query.sql \
  --preview 20 \
  --output /tmp/bigquery_preview

# Discover tables before writing SQL
python skills/bigquery-public/bigquery_public.py \
  --list-tables isb-cgc.TCGA_bioclin_v0 \
  --output /tmp/bigquery_tables

# Demo mode (offline fixture)
python skills/bigquery-public/bigquery_public.py --demo --output /tmp/bigquery_demo

# Via ClawBio runner
python clawbio.py run bigquery --demo
python clawbio.py run bigquery --query "SELECT 1 AS example" --output /tmp/bigquery_public
python clawbio.py run bigquery --describe isb-cgc.TCGA_bioclin_v0.Clinical --output /tmp/bigquery_schema

Demo

To verify the skill works:

python clawbio.py run bigquery --demo

Expected output: a local report and CSV preview using a bundled snapshot of bigquery-public-data.samples.shakespeare.

Algorithm / Methodology

  1. Normalize query: strip comments, mask literals, reject non-read-only SQL.
  2. Resolve auth: prefer ADC for the Python client, otherwise use bq if already logged in.
  3. Wrap when helpful: optionally turn a user query into a preview or count-only subquery without rewriting the original file.
  4. Run safely: apply --max-bytes-billed, --max-rows, and optional dry-run.
  5. Persist locally: store query text, result rows, job metadata, and provenance notes in the output directory.

Key parameters:

  • Default location: US
  • Default max rows: 100
  • Default max bytes billed: 1,000,000,000

Example Queries

  • "Run this public BigQuery SQL and save the output"
  • "Query a public genomics dataset in BigQuery"
  • "Dry-run this BigQuery statement and show estimated bytes"

Output Structure

output_directory/
├── report.md
├── result.json
├── tables/
│   └── results.csv
└── reproducibility/
    ├── commands.sh
    ├── environment.yml
    ├── job_metadata.json
    ├── provenance.json
    └── query.sql

Dependencies

Required:

  • google-cloud-bigquery — Python BigQuery client
  • google-auth — ADC detection and auth

Optional:

  • bq CLI — fallback backend when ADC is missing

Safety

  • Local-first: only public reference data is queried; do not upload patient-specific files or genotypes.
  • Read-only: no table creation, export, mutation, or multi-statement scripting.
  • Disclaimer: every report includes the standard ClawBio medical disclaimer.
  • Cost control: dry-run and billed-byte caps are enabled by default.

Integration with Bio Orchestrator

This v1 skill is intended for explicit invocation through clawbio.py run bigquery. Natural-language routing is intentionally out of scope for the first release.

Citations

04 同仓库更多 Skills

stanfish06
stanfish06/skillquarium

zarr-python

Stores and queries chunked N-D scientific arrays with Zarr-Python 3, including codecs, sharding, S3/GCS storage, and NumPy/Dask/Xarray integration. Use for array layout, bounded I/O, format migration, or scientific metadata preservation.

★ 12
stanfish06
stanfish06/skillquarium

yeet

Publish local changes to GitHub by confirming scope, committing intentionally, pushing the branch, and opening a draft PR through the GitHub app from this plugin, with `gh` used only as a fallback where connector coverage is insufficient.

★ 12
stanfish06
stanfish06/skillquarium

xarray

N-dimensional labeled arrays and datasets in Python. Built on top of NumPy and Dask. It introduces labels in the form of dimensions, coordinates, and attributes on top of raw NumPy-like arrays, making data analysis in physical sciences more intuitive and less error-prone. Use for working with multi-dimensional scientific data, NetCDF/GRIB/Zarr files, climate/weather/oceanographic datasets, remote sensing, geospatial imaging, large out-of-memory datasets with Dask, and labeled array operations.

★ 12
stanfish06
stanfish06/skillquarium

writing-mstest-tests

Use when asked to fix, rewrite, update, improve, modernize, show corrected code for, or explain existing MSTest tests or MSTest-specific configuration. Use for "review" when corrected code or edits are wanted, even for one pasted assertion or passing tests with bad failure output. Covers expected/actual labels; generic Boolean, collection, string, numeric, null, identity, exception, hard-cast, and object[] checks; TestContext/lifecycle; timeout/cancellation; OS/CI conditions, retry, cleanup, parallelization, MSTest.Sdk project setup, and MSTESTxxxx. Honor the installed MSTest version. DO NOT USE to design new test cases (code-testing), perform report-only audits, create project files rather than explain MSTest setup, run tests, migrate frameworks, or handle non-MSTest/non-.NET code.

★ 12
stanfish06
stanfish06/skillquarium

wgs-prs

End-to-end WGS to polygenic risk score pipeline. Takes paired-end FASTQ files (or a pre-existing VCF) through nf-core/sarek for variant calling, applies VCF QC (normalisation, hard filtering, Ti/Tv and Het/Hom checks), then computes polygenic risk scores via the PGS Catalog. Fills the FASTQ to VCF gap upstream of the gwas-prs skill.

★ 12
stanfish06
stanfish06/skillquarium

webapp-testing

Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.

★ 12