[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6XFT5M1YTMM2FHVVCG1JR91":3,"skill-same:01M6XFT5M1YTMM2FHVVCG1JR91":135},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":31,"publishedAt":52,"linkOnly":53,"markdownContent":54,"markdownBody":55,"frontMatter":56,"refName":67,"commitSha":68,"skillDirectory":69,"forksCount":14,"repositoryHtmlUrl":70,"repositoryHomepageUrl":6,"ownerHtmlUrl":71,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":72,"install":73,"resources":86},"01M6XFT5M1YTMM2FHVVCG1JR91","hugging-face-datasets",null,"Create and manage datasets on Hugging Face Hub. Supports initializing repos, defining configs\u002Fsystem prompts, streaming row updates, and SQL-based dataset querying\u002Ftransformation. Designed to work alongside HF MCP server for comprehensive dataset workflows.","fadrienne\u002FMyGitHub","fadrienne","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F197874571?v=4","01M6X64V8VWCRSMFZ4EPS2BWXG","Francesca Adrienne","01M6X64VD5MW4F9PHDWQKS7ZAF",0,"MIT","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fhugging-face-datasets\u002FSKILL.md",[18,23,26],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000010","后端与接口","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",{"categoryNo":27,"categoryName":28,"taxonomyType":29,"taxonomyTypeDesc":30},"01JBSEEDCAT000000000000033","数据分析师","PROFESSION","适用职业",[32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51],"face","hugging face datasets","hugging face","datasets","hf","for","hf mcp","transformation","duckdb","manage","hub","MCP","data-engineering","system","HuggingFace","server","streaming","create","repos","workflows","2026-11-10 10:12:30",false,"---\nname: hugging-face-datasets\ndescription: Create and manage datasets on Hugging Face Hub. Supports initializing repos, defining configs\u002Fsystem prompts, streaming row updates, and SQL-based dataset querying\u002Ftransformation. Designed to work alongside HF MCP server for comprehensive dataset workflows.\ncategory: data-engineering\nversion: 2.1.0\nauthor: Synthetic Sciences\nlicense: MIT\ntags: [Hugging Face, Datasets, Data Loading, Data Processing]\ndependencies: [huggingface-hub, duckdb, datasets, pandas]\n---\n\n# Overview\nThis skill provides tools to manage datasets on the Hugging Face Hub with a focus on creation, configuration, content management, and SQL-based data manipulation. It is designed to complement the existing Hugging Face MCP server by providing dataset editing and querying capabilities.\n\n## Integration with HF MCP Server\n- **Use HF MCP Server for**: Dataset discovery, search, and metadata retrieval\n- **Use This Skill for**: Dataset creation, content editing, SQL queries, data transformation, and structured data formatting\n\n# Version\n2.1.0\n\n# Dependencies\n# This skill uses PEP 723 scripts with inline dependency management\n# Scripts auto-install requirements when run with: uv run scripts\u002Fscript_name.py\n\n- uv (Python package manager)\n- Getting Started: See \"Usage Instructions\" below for PEP 723 usage\n\n# Core Capabilities\n\n## 1. Dataset Lifecycle Management\n- **Initialize**: Create new dataset repositories with proper structure\n- **Configure**: Store detailed configuration including system prompts and metadata\n- **Stream Updates**: Add rows efficiently without downloading entire datasets\n\n## 2. SQL-Based Dataset Querying (NEW)\nQuery any Hugging Face dataset using DuckDB SQL via `scripts\u002Fsql_manager.py`:\n- **Direct Queries**: Run SQL on datasets using the `hf:\u002F\u002F` protocol\n- **Schema Discovery**: Describe dataset structure and column types\n- **Data Sampling**: Get random samples for exploration\n- **Aggregations**: Count, histogram, unique values analysis\n- **Transformations**: Filter, join, reshape data with SQL\n- **Export & Push**: Save results locally or push to new Hub repos\n\n## 3. Multi-Format Dataset Support\nSupports diverse dataset types through template system:\n- **Chat\u002FConversational**: Chat templating, multi-turn dialogues, tool usage examples\n- **Text Classification**: Sentiment analysis, intent detection, topic classification\n- **Question-Answering**: Reading comprehension, factual QA, knowledge bases\n- **Text Completion**: Language modeling, code completion, creative writing\n- **Tabular Data**: Structured data for regression\u002Fclassification tasks\n- **Custom Formats**: Flexible schema definition for specialized needs\n\n## 4. Quality Assurance Features\n- **JSON Validation**: Ensures data integrity during uploads\n- **Batch Processing**: Efficient handling of large datasets\n- **Error Recovery**: Graceful handling of upload failures and conflicts\n\n# Usage Instructions\n\nThe skill includes two Python scripts that use PEP 723 inline dependency management:\n\n> **All paths are relative to the directory containing this SKILL.md\nfile.**\n> Scripts are run with: `uv run scripts\u002Fscript_name.py [arguments]`\n\n- `scripts\u002Fdataset_manager.py` - Dataset creation and management\n- `scripts\u002Fsql_manager.py` - SQL-based dataset querying and transformation\n\n### Prerequisites\n- `uv` package manager installed\n- `HF_TOKEN` environment variable must be set with a Write-access token\n\n---\n\n# SQL Dataset Querying (sql_manager.py)\n\nQuery, transform, and push Hugging Face datasets using DuckDB SQL. The `hf:\u002F\u002F` protocol provides direct access to any public dataset (or private with token).\n\n\n## Credential Setup\n\nHuggingFace token is auto-injected by openscience when connected via the dashboard.\n\n```bash\n# Verify credentials\n[ -n \"$HF_TOKEN\" ] && echo \"HF_TOKEN set\" || echo \"NOT SET\"\n```\n\nIf not set: connect HuggingFace at https:\u002F\u002Fapp.syntheticsciences.ai -> Services, then restart openscience.\n\n## Quick Start\n\n```bash\n# Query a dataset\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data WHERE subject='nutrition' LIMIT 10\"\n\n# Get dataset schema\nuv run scripts\u002Fsql_manager.py describe --dataset \"cais\u002Fmmlu\"\n\n# Sample random rows\nuv run scripts\u002Fsql_manager.py sample --dataset \"cais\u002Fmmlu\" --n 5\n\n# Count rows with filter\nuv run scripts\u002Fsql_manager.py count --dataset \"cais\u002Fmmlu\" --where \"subject='nutrition'\"\n```\n\n## SQL Query Syntax\n\nUse `data` as the table name in your SQL - it gets replaced with the actual `hf:\u002F\u002F` path:\n\n```sql\n-- Basic select\nSELECT * FROM data LIMIT 10\n\n-- Filtering\nSELECT * FROM data WHERE subject='nutrition'\n\n-- Aggregations\nSELECT subject, COUNT(*) as cnt FROM data GROUP BY subject ORDER BY cnt DESC\n\n-- Column selection and transformation\nSELECT question, choices[answer] AS correct_answer FROM data\n\n-- Regex matching\nSELECT * FROM data WHERE regexp_matches(question, 'nutrition|diet')\n\n-- String functions\nSELECT regexp_replace(question, '\\n', '') AS cleaned FROM data\n```\n\n## Common Operations\n\n### 1. Explore Dataset Structure\n```bash\n# Get schema\nuv run scripts\u002Fsql_manager.py describe --dataset \"cais\u002Fmmlu\"\n\n# Get unique values in column\nuv run scripts\u002Fsql_manager.py unique --dataset \"cais\u002Fmmlu\" --column \"subject\"\n\n# Get value distribution\nuv run scripts\u002Fsql_manager.py histogram --dataset \"cais\u002Fmmlu\" --column \"subject\" --bins 20\n```\n\n### 2. Filter and Transform\n```bash\n# Complex filtering with SQL\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT subject, COUNT(*) as cnt FROM data GROUP BY subject HAVING cnt > 100\"\n\n# Using transform command\nuv run scripts\u002Fsql_manager.py transform \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --select \"subject, COUNT(*) as cnt\" \\\n  --group-by \"subject\" \\\n  --order-by \"cnt DESC\" \\\n  --limit 10\n```\n\n### 3. Create Subsets and Push to Hub\n```bash\n# Query and push to new dataset\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data WHERE subject='nutrition'\" \\\n  --push-to \"username\u002Fmmlu-nutrition-subset\" \\\n  --private\n\n# Transform and push\nuv run scripts\u002Fsql_manager.py transform \\\n  --dataset \"ibm\u002Fduorc\" \\\n  --config \"ParaphraseRC\" \\\n  --select \"question, answers\" \\\n  --where \"LENGTH(question) > 50\" \\\n  --push-to \"username\u002Fduorc-long-questions\"\n```\n\n### 4. Export to Local Files\n```bash\n# Export to Parquet\nuv run scripts\u002Fsql_manager.py export \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data WHERE subject='nutrition'\" \\\n  --output \"nutrition.parquet\" \\\n  --format parquet\n\n# Export to JSONL\nuv run scripts\u002Fsql_manager.py export \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data LIMIT 100\" \\\n  --output \"sample.jsonl\" \\\n  --format jsonl\n```\n\n### 5. Working with Dataset Configs\u002FSplits\n```bash\n# Specify config (subset)\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"ibm\u002Fduorc\" \\\n  --config \"ParaphraseRC\" \\\n  --sql \"SELECT * FROM data LIMIT 5\"\n\n# Specify split\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --split \"test\" \\\n  --sql \"SELECT COUNT(*) FROM data\"\n\n# Query all splits\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --split \"*\" \\\n  --sql \"SELECT * FROM data LIMIT 10\"\n```\n\n### 6. Raw SQL with Full Paths\nFor complex queries or joining datasets:\n```bash\nuv run scripts\u002Fsql_manager.py raw --sql \"\n  SELECT a.*, b.* \n  FROM 'hf:\u002F\u002Fdatasets\u002Fdataset1@~parquet\u002Fdefault\u002Ftrain\u002F*.parquet' a\n  JOIN 'hf:\u002F\u002Fdatasets\u002Fdataset2@~parquet\u002Fdefault\u002Ftrain\u002F*.parquet' b\n  ON a.id = b.id\n  LIMIT 100\n\"\n```\n\n## Python API Usage\n\n```python\nfrom sql_manager import HFDatasetSQL\n\nsql = HFDatasetSQL()\n\n# Query\nresults = sql.query(\"cais\u002Fmmlu\", \"SELECT * FROM data WHERE subject='nutrition' LIMIT 10\")\n\n# Get schema\nschema = sql.describe(\"cais\u002Fmmlu\")\n\n# Sample\nsamples = sql.sample(\"cais\u002Fmmlu\", n=5, seed=42)\n\n# Count\ncount = sql.count(\"cais\u002Fmmlu\", where=\"subject='nutrition'\")\n\n# Histogram\ndist = sql.histogram(\"cais\u002Fmmlu\", \"subject\")\n\n# Filter and transform\nresults = sql.filter_and_transform(\n    \"cais\u002Fmmlu\",\n    select=\"subject, COUNT(*) as cnt\",\n    group_by=\"subject\",\n    order_by=\"cnt DESC\",\n    limit=10\n)\n\n# Push to Hub\nurl = sql.push_to_hub(\n    \"cais\u002Fmmlu\",\n    \"username\u002Fnutrition-subset\",\n    sql=\"SELECT * FROM data WHERE subject='nutrition'\",\n    private=True\n)\n\n# Export locally\nsql.export_to_parquet(\"cais\u002Fmmlu\", \"output.parquet\", sql=\"SELECT * FROM data LIMIT 100\")\n\nsql.close()\n```\n\n## HF Path Format\n\nDuckDB uses the `hf:\u002F\u002F` protocol to access datasets:\n```\nhf:\u002F\u002Fdatasets\u002F{dataset_id}@{revision}\u002F{config}\u002F{split}\u002F*.parquet\n```\n\nExamples:\n- `hf:\u002F\u002Fdatasets\u002Fcais\u002Fmmlu@~parquet\u002Fdefault\u002Ftrain\u002F*.parquet`\n- `hf:\u002F\u002Fdatasets\u002Fibm\u002Fduorc@~parquet\u002FParaphraseRC\u002Ftest\u002F*.parquet`\n\nThe `@~parquet` revision provides auto-converted Parquet files for any dataset format.\n\n## Useful DuckDB SQL Functions\n\n```sql\n-- String functions\nLENGTH(column)                    -- String length\nregexp_replace(col, '\\n', '')     -- Regex replace\nregexp_matches(col, 'pattern')    -- Regex match\nLOWER(col), UPPER(col)           -- Case conversion\n\n-- Array functions  \nchoices[0]                        -- Array indexing (0-based)\narray_length(choices)             -- Array length\nunnest(choices)                   -- Expand array to rows\n\n-- Aggregations\nCOUNT(*), SUM(col), AVG(col)\nGROUP BY col HAVING condition\n\n-- Sampling\nUSING SAMPLE 10                   -- Random sample\nUSING SAMPLE 10 (RESERVOIR, 42)   -- Reproducible sample\n\n-- Window functions\nROW_NUMBER() OVER (PARTITION BY col ORDER BY col2)\n```\n\n---\n\n# Dataset Creation (dataset_manager.py)\n\n### Recommended Workflow\n\n**1. Discovery (Use HF MCP Server):**\n```python\n# Use HF MCP tools to find existing datasets\nsearch_datasets(\"conversational AI training\")\nget_dataset_details(\"username\u002Fdataset-name\")\n```\n\n**2. Creation (Use This Skill):**\n```bash\n# Initialize new dataset\nuv run scripts\u002Fdataset_manager.py init --repo_id \"your-username\u002Fdataset-name\" [--private]\n\n# Configure with detailed system prompt\nuv run scripts\u002Fdataset_manager.py config --repo_id \"your-username\u002Fdataset-name\" --system_prompt \"$(cat system_prompt.txt)\"\n```\n\n**3. Content Management (Use This Skill):**\n```bash\n# Quick setup with any template\nuv run scripts\u002Fdataset_manager.py quick_setup \\\n  --repo_id \"your-username\u002Fdataset-name\" \\\n  --template classification\n\n# Add data with template validation\nuv run scripts\u002Fdataset_manager.py add_rows \\\n  --repo_id \"your-username\u002Fdataset-name\" \\\n  --template qa \\\n  --rows_json \"$(cat your_qa_data.json)\"\n```\n\n### Template-Based Data Structures\n\n**1. Chat Template (`--template chat`)**\n```json\n{\n  \"messages\": [\n    {\"role\": \"user\", \"content\": \"Natural user request\"},\n    {\"role\": \"assistant\", \"content\": \"Response with tool usage\"},\n    {\"role\": \"tool\", \"content\": \"Tool response\", \"tool_call_id\": \"call_123\"}\n  ],\n  \"scenario\": \"Description of use case\",\n  \"complexity\": \"simple|intermediate|advanced\"\n}\n```\n\n**2. Classification Template (`--template classification`)**\n```json\n{\n  \"text\": \"Input text to be classified\",\n  \"label\": \"classification_label\",\n  \"confidence\": 0.95,\n  \"metadata\": {\"domain\": \"technology\", \"language\": \"en\"}\n}\n```\n\n**3. QA Template (`--template qa`)**\n```json\n{\n  \"question\": \"What is the question being asked?\",\n  \"answer\": \"The complete answer\",\n  \"context\": \"Additional context if needed\",\n  \"answer_type\": \"factual|explanatory|opinion\",\n  \"difficulty\": \"easy|medium|hard\"\n}\n```\n\n**4. Completion Template (`--template completion`)**\n```json\n{\n  \"prompt\": \"The beginning text or context\",\n  \"completion\": \"The expected continuation\",\n  \"domain\": \"code|creative|technical|conversational\",\n  \"style\": \"description of writing style\"\n}\n```\n\n**5. Tabular Template (`--template tabular`)**\n```json\n{\n  \"columns\": [\n    {\"name\": \"feature1\", \"type\": \"numeric\", \"description\": \"First feature\"},\n    {\"name\": \"target\", \"type\": \"categorical\", \"description\": \"Target variable\"}\n  ],\n  \"data\": [\n    {\"feature1\": 123, \"target\": \"class_a\"},\n    {\"feature1\": 456, \"target\": \"class_b\"}\n  ]\n}\n```\n\n### Advanced System Prompt Template\n\nFor high-quality training data generation:\n```text\nYou are an AI assistant expert at using MCP tools effectively.\n\n## MCP SERVER DEFINITIONS\n[Define available servers and tools]\n\n## TRAINING EXAMPLE STRUCTURE\n[Specify exact JSON schema for chat templating]\n\n## QUALITY GUIDELINES\n[Detail requirements for realistic scenarios, progressive complexity, proper tool usage]\n\n## EXAMPLE CATEGORIES\n[List development workflows, debugging scenarios, data management tasks]\n```\n\n### Example Categories & Templates\n\nThe skill includes diverse training examples beyond just MCP usage:\n\n**Available Example Sets:**\n- `training_examples.json` - MCP tool usage examples (debugging, project setup, database analysis)\n- `diverse_training_examples.json` - Broader scenarios including:\n  - **Educational Chat** - Explaining programming concepts, tutorials\n  - **Git Workflows** - Feature branches, version control guidance\n  - **Code Analysis** - Performance optimization, architecture review\n  - **Content Generation** - Professional writing, creative brainstorming\n  - **Codebase Navigation** - Legacy code exploration, systematic analysis\n  - **Conversational Support** - Problem-solving, technical discussions\n\n**Using Different Example Sets:**\n```bash\n# Add MCP-focused examples\nuv run scripts\u002Fdataset_manager.py add_rows --repo_id \"your-username\u002Fdataset-name\" \\\n  --rows_json \"$(cat examples\u002Ftraining_examples.json)\"\n\n# Add diverse conversational examples\nuv run scripts\u002Fdataset_manager.py add_rows --repo_id \"your-username\u002Fdataset-name\" \\\n  --rows_json \"$(cat examples\u002Fdiverse_training_examples.json)\"\n\n# Mix both for comprehensive training data\nuv run scripts\u002Fdataset_manager.py add_rows --repo_id \"your-username\u002Fdataset-name\" \\\n  --rows_json \"$(jq -s '.[0] + .[1]' examples\u002Ftraining_examples.json examples\u002Fdiverse_training_examples.json)\"\n```\n\n### Commands Reference\n\n**List Available Templates:**\n```bash\nuv run scripts\u002Fdataset_manager.py list_templates\n```\n\n**Quick Setup (Recommended):**\n```bash\nuv run scripts\u002Fdataset_manager.py quick_setup --repo_id \"your-username\u002Fdataset-name\" --template classification\n```\n\n**Manual Setup:**\n```bash\n# Initialize repository\nuv run scripts\u002Fdataset_manager.py init --repo_id \"your-username\u002Fdataset-name\" [--private]\n\n# Configure with system prompt\nuv run scripts\u002Fdataset_manager.py config --repo_id \"your-username\u002Fdataset-name\" --system_prompt \"Your prompt here\"\n\n# Add data with validation\nuv run scripts\u002Fdataset_manager.py add_rows \\\n  --repo_id \"your-username\u002Fdataset-name\" \\\n  --template qa \\\n  --rows_json '[{\"question\": \"What is AI?\", \"answer\": \"Artificial Intelligence...\"}]'\n```\n\n**View Dataset Statistics:**\n```bash\nuv run scripts\u002Fdataset_manager.py stats --repo_id \"your-username\u002Fdataset-name\"\n```\n\n### Error Handling\n- **Repository exists**: Script will notify and continue with configuration\n- **Invalid JSON**: Clear error message with parsing details\n- **Network issues**: Automatic retry for transient failures\n- **Token permissions**: Validation before operations begin\n\n---\n\n# Combined Workflow Examples\n\n## Example 1: Create Training Subset from Existing Dataset\n```bash\n# 1. Explore the source dataset\nuv run scripts\u002Fsql_manager.py describe --dataset \"cais\u002Fmmlu\"\nuv run scripts\u002Fsql_manager.py histogram --dataset \"cais\u002Fmmlu\" --column \"subject\"\n\n# 2. Query and create subset\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data WHERE subject IN ('nutrition', 'anatomy', 'clinical_knowledge')\" \\\n  --push-to \"username\u002Fmmlu-medical-subset\" \\\n  --private\n```\n\n## Example 2: Transform and Reshape Data\n```bash\n# Transform MMLU to QA format with correct answers extracted\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT question, choices[answer] as correct_answer, subject FROM data\" \\\n  --push-to \"username\u002Fmmlu-qa-format\"\n```\n\n## Example 3: Merge Multiple Dataset Splits\n```bash\n# Export multiple splits and combine\nuv run scripts\u002Fsql_manager.py export \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --split \"*\" \\\n  --output \"mmlu_all.parquet\"\n```\n\n## Example 4: Quality Filtering\n```bash\n# Filter for high-quality examples\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"squad\" \\\n  --sql \"SELECT * FROM data WHERE LENGTH(context) > 500 AND LENGTH(question) > 20\" \\\n  --push-to \"username\u002Fsquad-filtered\"\n```\n\n## Example 5: Create Custom Training Dataset\n```bash\n# 1. Query source data\nuv run scripts\u002Fsql_manager.py export \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT question, subject FROM data WHERE subject='nutrition'\" \\\n  --output \"nutrition_source.jsonl\" \\\n  --format jsonl\n\n# 2. Process with your pipeline (add answers, format, etc.)\n\n# 3. Push processed data\nuv run scripts\u002Fdataset_manager.py init --repo_id \"username\u002Fnutrition-training\"\nuv run scripts\u002Fdataset_manager.py add_rows \\\n  --repo_id \"username\u002Fnutrition-training\" \\\n  --template qa \\\n  --rows_json \"$(cat processed_data.json)\"\n```\n","# Overview\nThis skill provides tools to manage datasets on the Hugging Face Hub with a focus on creation, configuration, content management, and SQL-based data manipulation. It is designed to complement the existing Hugging Face MCP server by providing dataset editing and querying capabilities.\n\n## Integration with HF MCP Server\n- **Use HF MCP Server for**: Dataset discovery, search, and metadata retrieval\n- **Use This Skill for**: Dataset creation, content editing, SQL queries, data transformation, and structured data formatting\n\n# Version\n2.1.0\n\n# Dependencies\n# This skill uses PEP 723 scripts with inline dependency management\n# Scripts auto-install requirements when run with: uv run scripts\u002Fscript_name.py\n\n- uv (Python package manager)\n- Getting Started: See \"Usage Instructions\" below for PEP 723 usage\n\n# Core Capabilities\n\n## 1. Dataset Lifecycle Management\n- **Initialize**: Create new dataset repositories with proper structure\n- **Configure**: Store detailed configuration including system prompts and metadata\n- **Stream Updates**: Add rows efficiently without downloading entire datasets\n\n## 2. SQL-Based Dataset Querying (NEW)\nQuery any Hugging Face dataset using DuckDB SQL via `scripts\u002Fsql_manager.py`:\n- **Direct Queries**: Run SQL on datasets using the `hf:\u002F\u002F` protocol\n- **Schema Discovery**: Describe dataset structure and column types\n- **Data Sampling**: Get random samples for exploration\n- **Aggregations**: Count, histogram, unique values analysis\n- **Transformations**: Filter, join, reshape data with SQL\n- **Export & Push**: Save results locally or push to new Hub repos\n\n## 3. Multi-Format Dataset Support\nSupports diverse dataset types through template system:\n- **Chat\u002FConversational**: Chat templating, multi-turn dialogues, tool usage examples\n- **Text Classification**: Sentiment analysis, intent detection, topic classification\n- **Question-Answering**: Reading comprehension, factual QA, knowledge bases\n- **Text Completion**: Language modeling, code completion, creative writing\n- **Tabular Data**: Structured data for regression\u002Fclassification tasks\n- **Custom Formats**: Flexible schema definition for specialized needs\n\n## 4. Quality Assurance Features\n- **JSON Validation**: Ensures data integrity during uploads\n- **Batch Processing**: Efficient handling of large datasets\n- **Error Recovery**: Graceful handling of upload failures and conflicts\n\n# Usage Instructions\n\nThe skill includes two Python scripts that use PEP 723 inline dependency management:\n\n> **All paths are relative to the directory containing this SKILL.md\nfile.**\n> Scripts are run with: `uv run scripts\u002Fscript_name.py [arguments]`\n\n- `scripts\u002Fdataset_manager.py` - Dataset creation and management\n- `scripts\u002Fsql_manager.py` - SQL-based dataset querying and transformation\n\n### Prerequisites\n- `uv` package manager installed\n- `HF_TOKEN` environment variable must be set with a Write-access token\n\n---\n\n# SQL Dataset Querying (sql_manager.py)\n\nQuery, transform, and push Hugging Face datasets using DuckDB SQL. The `hf:\u002F\u002F` protocol provides direct access to any public dataset (or private with token).\n\n\n## Credential Setup\n\nHuggingFace token is auto-injected by openscience when connected via the dashboard.\n\n```bash\n# Verify credentials\n[ -n \"$HF_TOKEN\" ] && echo \"HF_TOKEN set\" || echo \"NOT SET\"\n```\n\nIf not set: connect HuggingFace at https:\u002F\u002Fapp.syntheticsciences.ai -> Services, then restart openscience.\n\n## Quick Start\n\n```bash\n# Query a dataset\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data WHERE subject='nutrition' LIMIT 10\"\n\n# Get dataset schema\nuv run scripts\u002Fsql_manager.py describe --dataset \"cais\u002Fmmlu\"\n\n# Sample random rows\nuv run scripts\u002Fsql_manager.py sample --dataset \"cais\u002Fmmlu\" --n 5\n\n# Count rows with filter\nuv run scripts\u002Fsql_manager.py count --dataset \"cais\u002Fmmlu\" --where \"subject='nutrition'\"\n```\n\n## SQL Query Syntax\n\nUse `data` as the table name in your SQL - it gets replaced with the actual `hf:\u002F\u002F` path:\n\n```sql\n-- Basic select\nSELECT * FROM data LIMIT 10\n\n-- Filtering\nSELECT * FROM data WHERE subject='nutrition'\n\n-- Aggregations\nSELECT subject, COUNT(*) as cnt FROM data GROUP BY subject ORDER BY cnt DESC\n\n-- Column selection and transformation\nSELECT question, choices[answer] AS correct_answer FROM data\n\n-- Regex matching\nSELECT * FROM data WHERE regexp_matches(question, 'nutrition|diet')\n\n-- String functions\nSELECT regexp_replace(question, '\\n', '') AS cleaned FROM data\n```\n\n## Common Operations\n\n### 1. Explore Dataset Structure\n```bash\n# Get schema\nuv run scripts\u002Fsql_manager.py describe --dataset \"cais\u002Fmmlu\"\n\n# Get unique values in column\nuv run scripts\u002Fsql_manager.py unique --dataset \"cais\u002Fmmlu\" --column \"subject\"\n\n# Get value distribution\nuv run scripts\u002Fsql_manager.py histogram --dataset \"cais\u002Fmmlu\" --column \"subject\" --bins 20\n```\n\n### 2. Filter and Transform\n```bash\n# Complex filtering with SQL\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT subject, COUNT(*) as cnt FROM data GROUP BY subject HAVING cnt > 100\"\n\n# Using transform command\nuv run scripts\u002Fsql_manager.py transform \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --select \"subject, COUNT(*) as cnt\" \\\n  --group-by \"subject\" \\\n  --order-by \"cnt DESC\" \\\n  --limit 10\n```\n\n### 3. Create Subsets and Push to Hub\n```bash\n# Query and push to new dataset\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data WHERE subject='nutrition'\" \\\n  --push-to \"username\u002Fmmlu-nutrition-subset\" \\\n  --private\n\n# Transform and push\nuv run scripts\u002Fsql_manager.py transform \\\n  --dataset \"ibm\u002Fduorc\" \\\n  --config \"ParaphraseRC\" \\\n  --select \"question, answers\" \\\n  --where \"LENGTH(question) > 50\" \\\n  --push-to \"username\u002Fduorc-long-questions\"\n```\n\n### 4. Export to Local Files\n```bash\n# Export to Parquet\nuv run scripts\u002Fsql_manager.py export \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data WHERE subject='nutrition'\" \\\n  --output \"nutrition.parquet\" \\\n  --format parquet\n\n# Export to JSONL\nuv run scripts\u002Fsql_manager.py export \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data LIMIT 100\" \\\n  --output \"sample.jsonl\" \\\n  --format jsonl\n```\n\n### 5. Working with Dataset Configs\u002FSplits\n```bash\n# Specify config (subset)\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"ibm\u002Fduorc\" \\\n  --config \"ParaphraseRC\" \\\n  --sql \"SELECT * FROM data LIMIT 5\"\n\n# Specify split\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --split \"test\" \\\n  --sql \"SELECT COUNT(*) FROM data\"\n\n# Query all splits\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --split \"*\" \\\n  --sql \"SELECT * FROM data LIMIT 10\"\n```\n\n### 6. Raw SQL with Full Paths\nFor complex queries or joining datasets:\n```bash\nuv run scripts\u002Fsql_manager.py raw --sql \"\n  SELECT a.*, b.* \n  FROM 'hf:\u002F\u002Fdatasets\u002Fdataset1@~parquet\u002Fdefault\u002Ftrain\u002F*.parquet' a\n  JOIN 'hf:\u002F\u002Fdatasets\u002Fdataset2@~parquet\u002Fdefault\u002Ftrain\u002F*.parquet' b\n  ON a.id = b.id\n  LIMIT 100\n\"\n```\n\n## Python API Usage\n\n```python\nfrom sql_manager import HFDatasetSQL\n\nsql = HFDatasetSQL()\n\n# Query\nresults = sql.query(\"cais\u002Fmmlu\", \"SELECT * FROM data WHERE subject='nutrition' LIMIT 10\")\n\n# Get schema\nschema = sql.describe(\"cais\u002Fmmlu\")\n\n# Sample\nsamples = sql.sample(\"cais\u002Fmmlu\", n=5, seed=42)\n\n# Count\ncount = sql.count(\"cais\u002Fmmlu\", where=\"subject='nutrition'\")\n\n# Histogram\ndist = sql.histogram(\"cais\u002Fmmlu\", \"subject\")\n\n# Filter and transform\nresults = sql.filter_and_transform(\n    \"cais\u002Fmmlu\",\n    select=\"subject, COUNT(*) as cnt\",\n    group_by=\"subject\",\n    order_by=\"cnt DESC\",\n    limit=10\n)\n\n# Push to Hub\nurl = sql.push_to_hub(\n    \"cais\u002Fmmlu\",\n    \"username\u002Fnutrition-subset\",\n    sql=\"SELECT * FROM data WHERE subject='nutrition'\",\n    private=True\n)\n\n# Export locally\nsql.export_to_parquet(\"cais\u002Fmmlu\", \"output.parquet\", sql=\"SELECT * FROM data LIMIT 100\")\n\nsql.close()\n```\n\n## HF Path Format\n\nDuckDB uses the `hf:\u002F\u002F` protocol to access datasets:\n```\nhf:\u002F\u002Fdatasets\u002F{dataset_id}@{revision}\u002F{config}\u002F{split}\u002F*.parquet\n```\n\nExamples:\n- `hf:\u002F\u002Fdatasets\u002Fcais\u002Fmmlu@~parquet\u002Fdefault\u002Ftrain\u002F*.parquet`\n- `hf:\u002F\u002Fdatasets\u002Fibm\u002Fduorc@~parquet\u002FParaphraseRC\u002Ftest\u002F*.parquet`\n\nThe `@~parquet` revision provides auto-converted Parquet files for any dataset format.\n\n## Useful DuckDB SQL Functions\n\n```sql\n-- String functions\nLENGTH(column)                    -- String length\nregexp_replace(col, '\\n', '')     -- Regex replace\nregexp_matches(col, 'pattern')    -- Regex match\nLOWER(col), UPPER(col)           -- Case conversion\n\n-- Array functions  \nchoices[0]                        -- Array indexing (0-based)\narray_length(choices)             -- Array length\nunnest(choices)                   -- Expand array to rows\n\n-- Aggregations\nCOUNT(*), SUM(col), AVG(col)\nGROUP BY col HAVING condition\n\n-- Sampling\nUSING SAMPLE 10                   -- Random sample\nUSING SAMPLE 10 (RESERVOIR, 42)   -- Reproducible sample\n\n-- Window functions\nROW_NUMBER() OVER (PARTITION BY col ORDER BY col2)\n```\n\n---\n\n# Dataset Creation (dataset_manager.py)\n\n### Recommended Workflow\n\n**1. Discovery (Use HF MCP Server):**\n```python\n# Use HF MCP tools to find existing datasets\nsearch_datasets(\"conversational AI training\")\nget_dataset_details(\"username\u002Fdataset-name\")\n```\n\n**2. Creation (Use This Skill):**\n```bash\n# Initialize new dataset\nuv run scripts\u002Fdataset_manager.py init --repo_id \"your-username\u002Fdataset-name\" [--private]\n\n# Configure with detailed system prompt\nuv run scripts\u002Fdataset_manager.py config --repo_id \"your-username\u002Fdataset-name\" --system_prompt \"$(cat system_prompt.txt)\"\n```\n\n**3. Content Management (Use This Skill):**\n```bash\n# Quick setup with any template\nuv run scripts\u002Fdataset_manager.py quick_setup \\\n  --repo_id \"your-username\u002Fdataset-name\" \\\n  --template classification\n\n# Add data with template validation\nuv run scripts\u002Fdataset_manager.py add_rows \\\n  --repo_id \"your-username\u002Fdataset-name\" \\\n  --template qa \\\n  --rows_json \"$(cat your_qa_data.json)\"\n```\n\n### Template-Based Data Structures\n\n**1. Chat Template (`--template chat`)**\n```json\n{\n  \"messages\": [\n    {\"role\": \"user\", \"content\": \"Natural user request\"},\n    {\"role\": \"assistant\", \"content\": \"Response with tool usage\"},\n    {\"role\": \"tool\", \"content\": \"Tool response\", \"tool_call_id\": \"call_123\"}\n  ],\n  \"scenario\": \"Description of use case\",\n  \"complexity\": \"simple|intermediate|advanced\"\n}\n```\n\n**2. Classification Template (`--template classification`)**\n```json\n{\n  \"text\": \"Input text to be classified\",\n  \"label\": \"classification_label\",\n  \"confidence\": 0.95,\n  \"metadata\": {\"domain\": \"technology\", \"language\": \"en\"}\n}\n```\n\n**3. QA Template (`--template qa`)**\n```json\n{\n  \"question\": \"What is the question being asked?\",\n  \"answer\": \"The complete answer\",\n  \"context\": \"Additional context if needed\",\n  \"answer_type\": \"factual|explanatory|opinion\",\n  \"difficulty\": \"easy|medium|hard\"\n}\n```\n\n**4. Completion Template (`--template completion`)**\n```json\n{\n  \"prompt\": \"The beginning text or context\",\n  \"completion\": \"The expected continuation\",\n  \"domain\": \"code|creative|technical|conversational\",\n  \"style\": \"description of writing style\"\n}\n```\n\n**5. Tabular Template (`--template tabular`)**\n```json\n{\n  \"columns\": [\n    {\"name\": \"feature1\", \"type\": \"numeric\", \"description\": \"First feature\"},\n    {\"name\": \"target\", \"type\": \"categorical\", \"description\": \"Target variable\"}\n  ],\n  \"data\": [\n    {\"feature1\": 123, \"target\": \"class_a\"},\n    {\"feature1\": 456, \"target\": \"class_b\"}\n  ]\n}\n```\n\n### Advanced System Prompt Template\n\nFor high-quality training data generation:\n```text\nYou are an AI assistant expert at using MCP tools effectively.\n\n## MCP SERVER DEFINITIONS\n[Define available servers and tools]\n\n## TRAINING EXAMPLE STRUCTURE\n[Specify exact JSON schema for chat templating]\n\n## QUALITY GUIDELINES\n[Detail requirements for realistic scenarios, progressive complexity, proper tool usage]\n\n## EXAMPLE CATEGORIES\n[List development workflows, debugging scenarios, data management tasks]\n```\n\n### Example Categories & Templates\n\nThe skill includes diverse training examples beyond just MCP usage:\n\n**Available Example Sets:**\n- `training_examples.json` - MCP tool usage examples (debugging, project setup, database analysis)\n- `diverse_training_examples.json` - Broader scenarios including:\n  - **Educational Chat** - Explaining programming concepts, tutorials\n  - **Git Workflows** - Feature branches, version control guidance\n  - **Code Analysis** - Performance optimization, architecture review\n  - **Content Generation** - Professional writing, creative brainstorming\n  - **Codebase Navigation** - Legacy code exploration, systematic analysis\n  - **Conversational Support** - Problem-solving, technical discussions\n\n**Using Different Example Sets:**\n```bash\n# Add MCP-focused examples\nuv run scripts\u002Fdataset_manager.py add_rows --repo_id \"your-username\u002Fdataset-name\" \\\n  --rows_json \"$(cat examples\u002Ftraining_examples.json)\"\n\n# Add diverse conversational examples\nuv run scripts\u002Fdataset_manager.py add_rows --repo_id \"your-username\u002Fdataset-name\" \\\n  --rows_json \"$(cat examples\u002Fdiverse_training_examples.json)\"\n\n# Mix both for comprehensive training data\nuv run scripts\u002Fdataset_manager.py add_rows --repo_id \"your-username\u002Fdataset-name\" \\\n  --rows_json \"$(jq -s '.[0] + .[1]' examples\u002Ftraining_examples.json examples\u002Fdiverse_training_examples.json)\"\n```\n\n### Commands Reference\n\n**List Available Templates:**\n```bash\nuv run scripts\u002Fdataset_manager.py list_templates\n```\n\n**Quick Setup (Recommended):**\n```bash\nuv run scripts\u002Fdataset_manager.py quick_setup --repo_id \"your-username\u002Fdataset-name\" --template classification\n```\n\n**Manual Setup:**\n```bash\n# Initialize repository\nuv run scripts\u002Fdataset_manager.py init --repo_id \"your-username\u002Fdataset-name\" [--private]\n\n# Configure with system prompt\nuv run scripts\u002Fdataset_manager.py config --repo_id \"your-username\u002Fdataset-name\" --system_prompt \"Your prompt here\"\n\n# Add data with validation\nuv run scripts\u002Fdataset_manager.py add_rows \\\n  --repo_id \"your-username\u002Fdataset-name\" \\\n  --template qa \\\n  --rows_json '[{\"question\": \"What is AI?\", \"answer\": \"Artificial Intelligence...\"}]'\n```\n\n**View Dataset Statistics:**\n```bash\nuv run scripts\u002Fdataset_manager.py stats --repo_id \"your-username\u002Fdataset-name\"\n```\n\n### Error Handling\n- **Repository exists**: Script will notify and continue with configuration\n- **Invalid JSON**: Clear error message with parsing details\n- **Network issues**: Automatic retry for transient failures\n- **Token permissions**: Validation before operations begin\n\n---\n\n# Combined Workflow Examples\n\n## Example 1: Create Training Subset from Existing Dataset\n```bash\n# 1. Explore the source dataset\nuv run scripts\u002Fsql_manager.py describe --dataset \"cais\u002Fmmlu\"\nuv run scripts\u002Fsql_manager.py histogram --dataset \"cais\u002Fmmlu\" --column \"subject\"\n\n# 2. Query and create subset\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data WHERE subject IN ('nutrition', 'anatomy', 'clinical_knowledge')\" \\\n  --push-to \"username\u002Fmmlu-medical-subset\" \\\n  --private\n```\n\n## Example 2: Transform and Reshape Data\n```bash\n# Transform MMLU to QA format with correct answers extracted\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT question, choices[answer] as correct_answer, subject FROM data\" \\\n  --push-to \"username\u002Fmmlu-qa-format\"\n```\n\n## Example 3: Merge Multiple Dataset Splits\n```bash\n# Export multiple splits and combine\nuv run scripts\u002Fsql_manager.py export \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --split \"*\" \\\n  --output \"mmlu_all.parquet\"\n```\n\n## Example 4: Quality Filtering\n```bash\n# Filter for high-quality examples\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"squad\" \\\n  --sql \"SELECT * FROM data WHERE LENGTH(context) > 500 AND LENGTH(question) > 20\" \\\n  --push-to \"username\u002Fsquad-filtered\"\n```\n\n## Example 5: Create Custom Training Dataset\n```bash\n# 1. Query source data\nuv run scripts\u002Fsql_manager.py export \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT question, subject FROM data WHERE subject='nutrition'\" \\\n  --output \"nutrition_source.jsonl\" \\\n  --format jsonl\n\n# 2. Process with your pipeline (add answers, format, etc.)\n\n# 3. Push processed data\nuv run scripts\u002Fdataset_manager.py init --repo_id \"username\u002Fnutrition-training\"\nuv run scripts\u002Fdataset_manager.py add_rows \\\n  --repo_id \"username\u002Fnutrition-training\" \\\n  --template qa \\\n  --rows_json \"$(cat processed_data.json)\"\n```\n",{"name":5,"tags":57,"author":62,"license":15,"version":63,"category":44,"description":7,"dependencies":64},[58,59,60,61],"Hugging Face","Datasets","Data Loading","Data Processing","Synthetic Sciences","2.1.0",[65,40,35,66],"huggingface-hub","pandas","main","4aa39efd1cdab0b4657ecee16c92a92011a5b10f","openscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fhugging-face-datasets","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub","https:\u002F\u002Fgithub.com\u002Ffadrienne","2026-10-28 00:43:09",{"source":8,"skillName":5,"commands":74,"prompt":84,"zipAvailable":85},[75,78,81],{"tool":76,"command":77},"npx","npx skills add fadrienne\u002FMyGitHub --skill hugging-face-datasets",{"tool":79,"command":80},"bunx","bunx skills add fadrienne\u002FMyGitHub --skill hugging-face-datasets",{"tool":82,"command":83},"pnpm","pnpm dlx skills add fadrienne\u002FMyGitHub --skill hugging-face-datasets","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fhugging-face-datasets\u002FSKILL.md",true,[87,93,97,101,107,111,115,119,123,127,131],{"path":88,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":91,"sourceHtmlUrl":92},"examples\u002Fdiverse_training_examples.json","REFERENCE","参考文档",17797,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fhugging-face-datasets\u002Fexamples\u002Fdiverse_training_examples.json",{"path":94,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":95,"sourceHtmlUrl":96},"examples\u002Fsystem_prompt_template.txt",5451,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fhugging-face-datasets\u002Fexamples\u002Fsystem_prompt_template.txt",{"path":98,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":99,"sourceHtmlUrl":100},"examples\u002Ftraining_examples.json",6812,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fhugging-face-datasets\u002Fexamples\u002Ftraining_examples.json",{"path":102,"resourceType":103,"resourceTypeDesc":104,"sizeBytes":105,"sourceHtmlUrl":106},"scripts\u002Fdataset_manager.py","SCRIPT","脚本",18436,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fhugging-face-datasets\u002Fscripts\u002Fdataset_manager.py",{"path":108,"resourceType":103,"resourceTypeDesc":104,"sizeBytes":109,"sourceHtmlUrl":110},"scripts\u002Fsql_manager.py",28645,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fhugging-face-datasets\u002Fscripts\u002Fsql_manager.py",{"path":112,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":113,"sourceHtmlUrl":114},"templates\u002Fchat.json",2290,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fhugging-face-datasets\u002Ftemplates\u002Fchat.json",{"path":116,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":117,"sourceHtmlUrl":118},"templates\u002Fclassification.json",1923,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fhugging-face-datasets\u002Ftemplates\u002Fclassification.json",{"path":120,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":121,"sourceHtmlUrl":122},"templates\u002Fcompletion.json",2664,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fhugging-face-datasets\u002Ftemplates\u002Fcompletion.json",{"path":124,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":125,"sourceHtmlUrl":126},"templates\u002Fcustom.json",2412,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fhugging-face-datasets\u002Ftemplates\u002Fcustom.json",{"path":128,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":129,"sourceHtmlUrl":130},"templates\u002Fqa.json",2673,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fhugging-face-datasets\u002Ftemplates\u002Fqa.json",{"path":132,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":133,"sourceHtmlUrl":134},"templates\u002Ftabular.json",2546,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fhugging-face-datasets\u002Ftemplates\u002Ftabular.json",[136,164,190,222,249,276],{"skillNo":137,"name":138,"nameZh":6,"description":139,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":140,"categories":141,"keywords":143,"publishedAt":52},"01M7167B28PK20HP7R7ZED2DY0","statistical-power","Sample size calculation, power analysis, and effect size conversion for scientific studies. Use BEFORE data collection (a priori) to determine required N, or after (post-hoc) to compute achieved power. Covers parametric tests (t-test, ANOVA, chi-square, proportions, correlation, regression), simulation-based power for complex\u002Fclustered\u002Fnested designs, and power curves. Use this skill when someone asks 'how many samples do I need', 'what is my power', 'what effect size can I detect', or any variant. Pairs with \u002Fexperimental-design (choose design first, then compute power) and \u002Fstatistical-analysis (analyze data after collection).","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-power\u002FSKILL.md",[142],{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},[144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163],"power","statistical power","effect","scipy","regression","choose","design","uv","when","Bash","collection","analysis","calculation","ANOVA","conversion","analyze","t-test","chi-square","simulation","sample",{"skillNo":165,"name":166,"nameZh":6,"description":167,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":168,"categories":169,"keywords":173,"publishedAt":52},"01M71677EBCM8EPCE9BZCK8505","statistical-analysis","Guided statistical analysis with test selection and reporting. Use when you need help choosing appropriate tests for your data, assumption checking, power analysis, and APA-formatted results. Best for academic research reporting, test selection guidance. For implementing specific models programmatically use statsmodels.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-analysis\u002FSKILL.md",[170],{"categoryNo":171,"categoryName":172,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量",[155,174,144,175,176,177,152,178,179,180,37,181,182,183,184,185,186,187,188,189],"statistical analysis","academic research","implementing","selection","you","reporting","data","tests","use","with","test","research","results","statsmodels","models","help",{"skillNo":191,"name":192,"nameZh":6,"description":193,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":194,"categories":195,"keywords":203,"publishedAt":52},"01M71673R93XAHFH81DBNVNG7H","scientific-writing","Core skill for the deep research and writing tool. Write scientific manuscripts in full paragraphs (never bullet points). Use two-stage process with (1) section outlines with key points using research-lookup then (2) convert to flowing prose. IMRAD structure, citations (APA\u002FAMA\u002FVancouver), figures\u002Ftables, reporting guidelines (CONSORT\u002FSTROBE\u002FPRISMA), for research papers and journal submissions.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-writing\u002FSKILL.md",[196,199,200],{"categoryNo":197,"categoryName":198,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":201,"categoryName":202,"taxonomyType":29,"taxonomyTypeDesc":30},"01JBSEEDCAT000000000000036","内容创作者",[204,205,206,207,208,209,210,211,212,213,153,214,215,216,217,218,219,220,179,221],"writing","scientific","scientific writing","Citations","deep","deep research","never","papers","journal","false","api","prisma","structure","openrouter","Outlines","core skill","read","features",{"skillNo":223,"name":224,"nameZh":6,"description":225,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":226,"categories":227,"keywords":232,"publishedAt":52},"01M7166ZPKXBF4EBPHVTJPQBB1","scientific-visualization","Meta-skill for publication-ready figures. Use when creating journal submission figures requiring multi-panel layouts, significance annotations, error bars, colorblind-safe palettes, and specific journal formatting (Nature, Science, Cell). Orchestrates matplotlib\u002Fseaborn\u002Fplotly with publication styles. For quick exploration use seaborn or plotly directly.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-visualization\u002FSKILL.md",[228,231],{"categoryNo":229,"categoryName":230,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000013","分析与可视化",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},[233,205,234,235,236,237,238,239,240,212,152,241,242,243,244,245,246,247,37,248],"visualization","scientific visualization","error","ready","quick","exploration","meta-skill","Meta","safe","plotly","annotations","formatting","Publication","figures","panel","submission",{"skillNo":250,"name":251,"nameZh":6,"description":252,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":253,"categories":254,"keywords":261,"publishedAt":52},"01M7166VXJWQSSC3RQZP60WBGR","research-code","Debug, profile, review, and restructure Python research and analysis code. Use when analysis scripts crash, produce wrong results, run too slowly, or need a correctness review. Covers: reading tracebacks and diagnosing root causes, hypothesis-driven debugging, reviewing code for common statistical\u002Fnumerical errors, profiling data pipelines, improving reproducibility (seeds, environments, determinism), and structuring analysis scripts for sharing. Distinct from \u002Fexploratory-data-analysis (which understands data) and \u002Fstatistical-analysis (which chooses and runs tests) — this skill focuses on the code itself.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fresearch-code\u002FSKILL.md",[255,258],{"categoryNo":256,"categoryName":257,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000021","资料检索与研究",{"categoryNo":259,"categoryName":260,"taxonomyType":29,"taxonomyTypeDesc":30},"01JBSEEDCAT000000000000030","开发工程师",[262,263,185,264,265,266,267,151,268,152,153,155,269,270,271,272,273,274,220,275],"code","research code","which","memory","data-analysis","errors","debugging","common","review","data pipelines","run","scripts","line","write",{"skillNo":277,"name":278,"nameZh":6,"description":279,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":280,"licenseFileUrl":6,"sourceHtmlUrl":281,"categories":282,"keywords":289,"publishedAt":52},"01M7166CF5HSR7SVZYZFWTDX55","markdown-mermaid-writing","Comprehensive markdown and Mermaid diagram writing skill. Use when creating any scientific document, report, analysis, or visualization. Establishes text-based diagrams as the default documentation standard with full style guides (markdown + mermaid), 24 diagram type references, and 9 document templates.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fmarkdown-mermaid-writing\u002FSKILL.md",[283,286,287,288],{"categoryNo":284,"categoryName":285,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000002","文档处理",{"categoryNo":197,"categoryName":198,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":201,"categoryName":202,"taxonomyType":29,"taxonomyTypeDesc":30},[204,290,291,292,293,294,233,295,296,152,153,155,297,298,299,300,301,302,205,220],"markdown mermaid writing","mermaid","markdown","community","diagrams","project","default","team","guides","text","documentation","references","diagram"]