text-and-datasets
Use Yellowbrick dataset loaders, cache controls, and text visualizers for tabular data, corpora, and text embeddings.
- License 许可
- Apache-2.0
- 最近更新
- 2026-09-03
- Stars
- ★ 351
- Forks
- 27
01 前言 Front Matter
- name
- text-and-datasets
- license
- Apache 2.0
- metadata
- { "disco-role": "operating" }
- description
- Use Yellowbrick dataset loaders, cache controls, and text visualizers for tabular data, corpora, and text embeddings.
- disable-model-invocation
- true
02 资源文件 5
- 参考文档references/datasets.md9.6 KB
- 参考文档references/text-visualizers.md10.3 KB
- 参考文档references/troubleshooting.md11.0 KB
- 脚本scripts/check_dataset_cache.py8.6 KB
- 脚本scripts/text_smoke.py6.2 KB
03 SKILL.md 正文
Yellowbrick Text and Datasets
Use this sub-skill when the task is about Yellowbrick dataset loading, dataset
cache/download behavior, the load_* helpers, the hobbies corpus, or
text-specific visualizers. It covers FreqDistVisualizer, TSNEVisualizer,
UMAPVisualizer, DispersionPlot, WordCorrelationPlot, and
PosTagVisualizer.
For package-wide Yellowbrick lifecycle, axes reuse, style, and headless saving, read shared visualizer patterns. For broad install, Matplotlib backend/font, and scikit-learn compatibility failures, read the root troubleshooting reference before applying text/dataset-specific fixes.
Use this sub-skill for
- Choosing among Yellowbrick example datasets and matching them to regression, classification, clustering, time-series-style, or text-analysis tasks.
- Loading tabular datasets with
load_bikeshare(),load_concrete(),load_credit(),load_energy(),load_game(),load_mushroom(),load_occupancy(),load_spam(),load_walking(), orload_nfl(). - Loading the hobbies corpus with
load_hobbies()and usingCorpus.data,Corpus.target,Corpus.labels, orCorpus.files. - Deciding between default
X, yreturns,return_dataset=True,to_numpy(),to_pandas(), andto_dataframe(). - Controlling or inspecting the dataset cache with
data_home,YELLOWBRICK_DATA, and the no-download cache inspector. - Using text visualizers after the corpus is already vectorized, tokenized, or part-of-speech tagged.
- Explaining optional extras:
pandasfor DataFrame returns,umap-learnforUMAPVisualizer, andnltk/spacyplus data/models for parser-based POS tagging.
Route elsewhere when
- The user has already loaded data and now wants classifier score plots such as reports, confusion matrices, ROC/PR curves, or threshold tuning: route to classifier visualizers.
- The user wants residuals, prediction error, Cook's distance, or alpha selection: route to regressor visualizers.
- The user wants feature ranking, PCA/manifold feature projections, class balance, binning, or feature-target correlation: route to feature-target visualizers.
- The user wants clustering diagnostics, validation/learning curves, CV scores, RFECV, feature importances, or dropping curves: route to cluster-model-selection.
- The user asks about contrib wrappers, decision boundaries, missing-value contrib plots, or statsmodels adapters: route to contrib-and-extensions.
Required read and run map
| Need | Read or run | When to use it |
|---|---|---|
Dataset names, task fit, load_*, return types, Dataset/Corpus, cache rules, downloader flags, and offline behavior |
datasets reference | Read before giving loader code, cache advice, or no-network guidance. |
| Text visualizer selection, exact input formats, optional extras, and safe file output | text visualizers reference | Read before using any yellowbrick.text visualizer or diagnosing text input-shape errors. |
| Dataset cache/download/signature/offline failures, missing optional dependencies, wrong class names, parser/tagset errors, and display/font warnings | text/dataset troubleshooting | Read after any failure symptom or before advising a risky cleanup/download action. |
Common Yellowbrick fit/score/transform/show(outpath=...), Matplotlib Agg, axes, and style rules |
shared visualizer patterns | Read before writing report-quality code or headless automation. |
| Broad installation, backend, font, or scikit-learn compatibility issues | root troubleshooting | Read when the problem is not specific to text or datasets. |
| No-download cache inspection | check_dataset_cache.py | Run when the user is offline, wants to know what is already cached, or needs signature/cache state without downloads or deletes. |
| Safe text plotting smoke | text_smoke.py | Run to confirm yellowbrick.text can render inline synthetic text plots with Matplotlib Agg; it uses no external data and no optional downloads. |
Example safe runs from an environment where Yellowbrick is importable:
python skills/disco/yellowbrick/sub-skills/text-and-datasets/scripts/check_dataset_cache.py --dataset hobbies
python skills/disco/yellowbrick/sub-skills/text-and-datasets/scripts/text_smoke.py --outdir /tmp/yellowbrick-text-smoke
Dataset quick map
| Loader | Object | Shape or size | Best fit | Notes |
|---|---|---|---|---|
load_bikeshare() |
Dataset |
X=(17379, 12) |
regression | Bike sharing demand examples. |
load_concrete() |
Dataset |
X=(1030, 8) |
regression | Small, numeric, good for regressor demos. |
load_credit() |
Dataset |
X=(30000, 23) |
binary classification / clustering | Credit default data; can be large for quick CV examples. |
load_energy() |
Dataset |
X=(768, 8) |
regression / multi-output source data | Default y is the documented primary target. |
load_game() |
Dataset |
X=(67557, 42) |
multiclass classification | Categorical features usually need encoding before sklearn estimators. |
load_hobbies() |
Corpus |
448 docs, 5 labels |
text analysis / classification / clustering | Returns a Corpus, not X, y; use corpus.data and corpus.target. |
load_mushroom() |
Dataset |
X=(8123, 3) |
binary classification / clustering | Categorical mushroom attributes need preprocessing for many estimators. |
load_occupancy() |
Dataset |
X=(20560, 5) |
binary classification | Multivariate time-series-style occupancy data. |
load_spam() |
Dataset |
X=(4600, 57) |
binary classification / threshold analysis | Useful for threshold and imbalance demonstrations. |
load_walking() |
Dataset |
X=(149332, 4) |
clustering / time-series-style diagnostics | Large row count; sample for quick visual checks. |
load_nfl() |
Dataset |
X=(494, 23) |
clustering | Football receiver clustering examples. |
Text visualizer quick map
| Visualizer | Input required | Best fit | Constraints |
|---|---|---|---|
FreqDistVisualizer |
Count-vectorized document matrix plus feature names | Top-token counts in a corpus or by class | Use FreqDistVisualizer, not FrequencyVisualizer, in public guidance. |
TSNEVisualizer |
Numeric vectorized corpus matrix | 2D document embedding and class/cluster separation | t-SNE is expensive; vectorize first and lower perplexity for tiny samples. |
UMAPVisualizer |
Numeric vectorized corpus matrix | 2D document embedding with UMAP | Requires optional umap-learn; use metric="cosine" for many text vectors. |
DispersionPlot |
Tokenized documents: list/generator of token lists | Lexical dispersion of known terms across documents | Search terms must survive tokenization and casing. |
WordCorrelationPlot |
Raw text documents: list/generator of strings | Word or phrase co-occurrence heatmap | Every requested word/phrase must appear in the corpus. |
PosTagVisualizer |
Pre-tagged nested docs or raw text with parser | POS tag distributions | Prefer pre-tagged tuples for no-optional-dependency smoke checks. |
Canonical dataset pattern
from yellowbrick.datasets import load_concrete, load_hobbies
# Tabular data: pandas DataFrame/Series if pandas is installed, otherwise numpy arrays.
X, y = load_concrete()
# Request the object when you need metadata or explicit return control.
dataset = load_concrete(return_dataset=True)
X_np, y_np = dataset.to_numpy()
# Text corpus loaders return Corpus objects, not X/y directly.
corpus = load_hobbies()
docs = corpus.data
target = corpus.target
labels = corpus.labels
Canonical text pattern
import matplotlib
matplotlib.use("Agg") # set before pyplot in headless sessions
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from yellowbrick.text import FreqDistVisualizer, TSNEVisualizer
count_vectorizer = CountVectorizer()
X_counts = count_vectorizer.fit_transform(docs)
features = count_vectorizer.get_feature_names_out()
freq = FreqDistVisualizer(features=features)
freq.fit(X_counts, target)
freq.show(outpath="freqdist.png", clear_figure=True)
X_tfidf = TfidfVectorizer().fit_transform(docs)
tsne = TSNEVisualizer(random_state=13, perplexity=10)
tsne.fit(X_tfidf, target)
tsne.show(outpath="tsne.png", clear_figure=True)
Safety checklist
- Do not call a loader in offline mode unless its dataset directory already exists in the selected cache; missing data triggers Yellowbrick's downloader.
- Use the cache inspector before telling an offline user that a loader will work.
- Use
return_dataset=Trueonly on tabular loaders;load_hobbies()already returns aCorpusobject. - Confirm whether
pandasis installed before promising DataFrame/Series returns; numpy arrays are the normal fallback. - Confirm text input state: raw strings for vectorizers and
WordCorrelationPlot, token lists forDispersionPlot, numeric vectors for TSNE/UMAP, and nested(token, tag)tuples for parser-free POS visualization. - Confirm optional dependencies before recommending
UMAPVisualizeror raw-text parser mode forPosTagVisualizer. - Save figures with a non-interactive backend and class visualizer API when the output must be a file.
04 同仓库更多 Skills
passnet-triton-opt
Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.
passnet-solve
END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.
passnet-skill
PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.
passnet-pattern-fusion
Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.
passnet-orchestrate
Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.
passnet-feedback
Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).