AGENT SKILL

text-and-datasets

Use Yellowbrick dataset loaders, cache controls, and text visualizers for tabular data, corpora, and text embeddings.

License 许可
Apache-2.0
最近更新
2026-09-03
Stars
★ 351
Forks
27

01 前言 Front Matter

name
text-and-datasets
license
Apache 2.0
metadata
{ "disco-role": "operating" }
description
Use Yellowbrick dataset loaders, cache controls, and text visualizers for tabular data, corpora, and text embeddings.
disable-model-invocation
true

02 资源文件 5

03 SKILL.md 正文

Yellowbrick Text and Datasets

Use this sub-skill when the task is about Yellowbrick dataset loading, dataset cache/download behavior, the load_* helpers, the hobbies corpus, or text-specific visualizers. It covers FreqDistVisualizer, TSNEVisualizer, UMAPVisualizer, DispersionPlot, WordCorrelationPlot, and PosTagVisualizer.

For package-wide Yellowbrick lifecycle, axes reuse, style, and headless saving, read shared visualizer patterns. For broad install, Matplotlib backend/font, and scikit-learn compatibility failures, read the root troubleshooting reference before applying text/dataset-specific fixes.

Use this sub-skill for

  • Choosing among Yellowbrick example datasets and matching them to regression, classification, clustering, time-series-style, or text-analysis tasks.
  • Loading tabular datasets with load_bikeshare(), load_concrete(), load_credit(), load_energy(), load_game(), load_mushroom(), load_occupancy(), load_spam(), load_walking(), or load_nfl().
  • Loading the hobbies corpus with load_hobbies() and using Corpus.data, Corpus.target, Corpus.labels, or Corpus.files.
  • Deciding between default X, y returns, return_dataset=True, to_numpy(), to_pandas(), and to_dataframe().
  • Controlling or inspecting the dataset cache with data_home, YELLOWBRICK_DATA, and the no-download cache inspector.
  • Using text visualizers after the corpus is already vectorized, tokenized, or part-of-speech tagged.
  • Explaining optional extras: pandas for DataFrame returns, umap-learn for UMAPVisualizer, and nltk/spacy plus data/models for parser-based POS tagging.

Route elsewhere when

  • The user has already loaded data and now wants classifier score plots such as reports, confusion matrices, ROC/PR curves, or threshold tuning: route to classifier visualizers.
  • The user wants residuals, prediction error, Cook's distance, or alpha selection: route to regressor visualizers.
  • The user wants feature ranking, PCA/manifold feature projections, class balance, binning, or feature-target correlation: route to feature-target visualizers.
  • The user wants clustering diagnostics, validation/learning curves, CV scores, RFECV, feature importances, or dropping curves: route to cluster-model-selection.
  • The user asks about contrib wrappers, decision boundaries, missing-value contrib plots, or statsmodels adapters: route to contrib-and-extensions.

Required read and run map

Need Read or run When to use it
Dataset names, task fit, load_*, return types, Dataset/Corpus, cache rules, downloader flags, and offline behavior datasets reference Read before giving loader code, cache advice, or no-network guidance.
Text visualizer selection, exact input formats, optional extras, and safe file output text visualizers reference Read before using any yellowbrick.text visualizer or diagnosing text input-shape errors.
Dataset cache/download/signature/offline failures, missing optional dependencies, wrong class names, parser/tagset errors, and display/font warnings text/dataset troubleshooting Read after any failure symptom or before advising a risky cleanup/download action.
Common Yellowbrick fit/score/transform/show(outpath=...), Matplotlib Agg, axes, and style rules shared visualizer patterns Read before writing report-quality code or headless automation.
Broad installation, backend, font, or scikit-learn compatibility issues root troubleshooting Read when the problem is not specific to text or datasets.
No-download cache inspection check_dataset_cache.py Run when the user is offline, wants to know what is already cached, or needs signature/cache state without downloads or deletes.
Safe text plotting smoke text_smoke.py Run to confirm yellowbrick.text can render inline synthetic text plots with Matplotlib Agg; it uses no external data and no optional downloads.

Example safe runs from an environment where Yellowbrick is importable:

python skills/disco/yellowbrick/sub-skills/text-and-datasets/scripts/check_dataset_cache.py --dataset hobbies
python skills/disco/yellowbrick/sub-skills/text-and-datasets/scripts/text_smoke.py --outdir /tmp/yellowbrick-text-smoke

Dataset quick map

Loader Object Shape or size Best fit Notes
load_bikeshare() Dataset X=(17379, 12) regression Bike sharing demand examples.
load_concrete() Dataset X=(1030, 8) regression Small, numeric, good for regressor demos.
load_credit() Dataset X=(30000, 23) binary classification / clustering Credit default data; can be large for quick CV examples.
load_energy() Dataset X=(768, 8) regression / multi-output source data Default y is the documented primary target.
load_game() Dataset X=(67557, 42) multiclass classification Categorical features usually need encoding before sklearn estimators.
load_hobbies() Corpus 448 docs, 5 labels text analysis / classification / clustering Returns a Corpus, not X, y; use corpus.data and corpus.target.
load_mushroom() Dataset X=(8123, 3) binary classification / clustering Categorical mushroom attributes need preprocessing for many estimators.
load_occupancy() Dataset X=(20560, 5) binary classification Multivariate time-series-style occupancy data.
load_spam() Dataset X=(4600, 57) binary classification / threshold analysis Useful for threshold and imbalance demonstrations.
load_walking() Dataset X=(149332, 4) clustering / time-series-style diagnostics Large row count; sample for quick visual checks.
load_nfl() Dataset X=(494, 23) clustering Football receiver clustering examples.

Text visualizer quick map

Visualizer Input required Best fit Constraints
FreqDistVisualizer Count-vectorized document matrix plus feature names Top-token counts in a corpus or by class Use FreqDistVisualizer, not FrequencyVisualizer, in public guidance.
TSNEVisualizer Numeric vectorized corpus matrix 2D document embedding and class/cluster separation t-SNE is expensive; vectorize first and lower perplexity for tiny samples.
UMAPVisualizer Numeric vectorized corpus matrix 2D document embedding with UMAP Requires optional umap-learn; use metric="cosine" for many text vectors.
DispersionPlot Tokenized documents: list/generator of token lists Lexical dispersion of known terms across documents Search terms must survive tokenization and casing.
WordCorrelationPlot Raw text documents: list/generator of strings Word or phrase co-occurrence heatmap Every requested word/phrase must appear in the corpus.
PosTagVisualizer Pre-tagged nested docs or raw text with parser POS tag distributions Prefer pre-tagged tuples for no-optional-dependency smoke checks.

Canonical dataset pattern

from yellowbrick.datasets import load_concrete, load_hobbies

# Tabular data: pandas DataFrame/Series if pandas is installed, otherwise numpy arrays.
X, y = load_concrete()

# Request the object when you need metadata or explicit return control.
dataset = load_concrete(return_dataset=True)
X_np, y_np = dataset.to_numpy()

# Text corpus loaders return Corpus objects, not X/y directly.
corpus = load_hobbies()
docs = corpus.data
target = corpus.target
labels = corpus.labels

Canonical text pattern

import matplotlib
matplotlib.use("Agg")  # set before pyplot in headless sessions

from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from yellowbrick.text import FreqDistVisualizer, TSNEVisualizer

count_vectorizer = CountVectorizer()
X_counts = count_vectorizer.fit_transform(docs)
features = count_vectorizer.get_feature_names_out()

freq = FreqDistVisualizer(features=features)
freq.fit(X_counts, target)
freq.show(outpath="freqdist.png", clear_figure=True)

X_tfidf = TfidfVectorizer().fit_transform(docs)
tsne = TSNEVisualizer(random_state=13, perplexity=10)
tsne.fit(X_tfidf, target)
tsne.show(outpath="tsne.png", clear_figure=True)

Safety checklist

  • Do not call a loader in offline mode unless its dataset directory already exists in the selected cache; missing data triggers Yellowbrick's downloader.
  • Use the cache inspector before telling an offline user that a loader will work.
  • Use return_dataset=True only on tabular loaders; load_hobbies() already returns a Corpus object.
  • Confirm whether pandas is installed before promising DataFrame/Series returns; numpy arrays are the normal fallback.
  • Confirm text input state: raw strings for vectorizers and WordCorrelationPlot, token lists for DispersionPlot, numeric vectors for TSNE/UMAP, and nested (token, tag) tuples for parser-free POS visualization.
  • Confirm optional dependencies before recommending UMAPVisualizer or raw-text parser mode for PosTagVisualizer.
  • Save figures with a non-interactive backend and class visualizer API when the output must be a file.

04 同仓库更多 Skills

VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-triton-opt

Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-solve

END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-skill

PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-pattern-fusion

Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-orchestrate

Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-feedback

Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).

★ 351