AGENT SKILL

dataset-domain-token-counts-via-metadata-config

Use when a HuggingFace dataset task asks for token counts or filtering by a named domain and the dataset has multiple configs, only one of which contains domain and token-text columns.

License 许可
Apache-2.0
所属仓库
RUC-NLPIR/FlashRSI
最近更新
2026-11-12
Stars
★ 394
Forks
23

01 前言 Front Matter

name
dataset-domain-token-counts-via-metadata-config
flashrsi
{ "level": "topic", "algorithm": "evo_harness", "source_task_ids": [ "count-dataset-tokens" ] }
description
Use when a HuggingFace dataset task asks for token counts or filtering by a named domain and the dataset has multiple configs, only one of which contains domain and token-text columns.

03 SKILL.md 正文

Use the metadata config for domain-filtered token counts on OpenThoughts-style datasets

When a dataset task asks for domain-level token counts, do not assume the default config has the needed columns. Inspect the available configs and load the one that actually holds domain labels and reasoning/solution text.

Procedure:

  1. List configs before loading:
    from datasets import get_dataset_config_names
    print(get_dataset_config_names("ryanmarten/OpenThoughts-1k-sample"))
    
  2. Read the dataset card or README for documented configs and domain groupings.
  3. Load the config with the domain and token-text columns, often metadata, instead of default:
    from datasets import load_dataset
    ds = load_dataset("ryanmarten/OpenThoughts-1k-sample", "metadata")
    
  4. Map README category names to the actual metadata domain values. For example, a Science category usually corresponds to physics, chemistry, and biology; do not search for a literal science domain value.
  5. Validate before counting:
    print(ds["train"].column_names)
    print(ds["train"].unique("domain"))
    

Why: on ryanmarten/OpenThoughts-1k-sample the default config only has system and conversations, while domain, deepseek_reasoning, and deepseek_solution live in the metadata config. Counting tokens by domain from default returns no domain labels and can silently count the wrong text.

Supporting evidence: count-dataset-tokens (this batch).

04 同仓库更多 Skills

RUC-NLPIR
RUC-NLPIR/FlashRSI

dynamic-resources

Example skill loaded from resources_discover

★ 394
RUC-NLPIR
RUC-NLPIR/FlashRSI

hermes-agent

Use the FlashRSI Hermes Snapshot's headless Python runtime, tools, sessions, and compression paths.

★ 394
RUC-NLPIR
RUC-NLPIR/FlashRSI

skill-installer

Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos).

★ 394
RUC-NLPIR
RUC-NLPIR/FlashRSI

skill-creator

Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations.

★ 394
RUC-NLPIR
RUC-NLPIR/FlashRSI

review-agent

Perform a read-only, defect-first review of a specified code change and return every actionable finding. Use when another agent delegates review of uncommitted changes, a base-branch diff, a commit, or custom review instructions.

★ 394
RUC-NLPIR
RUC-NLPIR/FlashRSI

plugin-creator

Create and scaffold plugin directories for Codex with a required `.codex-plugin/plugin.json`, optional plugin folders/files, valid manifest defaults, and personal-marketplace entries by default. Use when Codex needs to create a new personal plugin, add optional plugin structure, generate or update marketplace entries for plugin ordering and availability metadata, or update an existing local plugin during development with the CLI-driven cachebuster and reinstall flow.

★ 394