szl-dataset-readiness
Audits a tabular dataset before training or evaluation for exact duplicate rows across train/test, the same subject, patient, cell line or sensor appearing in both splits, labels observed after the prediction time, and preprocessing fitted on held-out rows, all bound to actual row IDs. Use when the user is about to split data, asks 'is my test set clean', reports suspiciously high held-out accuracy, or a reviewer asks about leakage. Not a data-cleaning or imputation tool; it does not approve scientific suitability or data rights.
- License 许可
- Apache-2.0
- 最近更新
- 2026-11-15
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- szl-dataset-readiness
- license
- Apache-2.0
- description
- Audits a tabular dataset before training or evaluation for exact duplicate rows across train/test, the same subject, patient, cell line or sensor appearing in both splits, labels observed after the prediction time, and preprocessing fitted on held-out rows, all bound to actual row IDs. Use when the user is about to split data, asks 'is my test set clean', reports suspiciously high held-out accuracy, or a reviewer asks about leakage. Not a data-cleaning or imputation tool; it does not approve scientific suitability or data rights.
02 资源文件 5
- 资产文件assets/example.json584 B
- 资产文件assets/leakage-clean.json1.1 KB
- 脚本kernel.py17.5 KB
- 参考文档references/contract.md5.1 KB
- 脚本scripts/run.py3.3 KB
03 SKILL.md 正文
Dataset readiness
Most impossible accuracies come from a test set that already knows the answer. This audit finds the mechanical versions of that mistake and names the rows involved. Python 3.10+, stdlib, offline.
Use when
- A patient, mouse, cell line, sensor or template family appears in both train and test.
- Two rows are byte-identical except for the split they landed in.
- A label (diagnosis, failure, churn) was recorded after the moment the prediction would have been made.
- A scaler, imputer or vocabulary was fitted on all rows and then "held-out" rows were scored.
Quick start
python scripts/run.py assets/example.json
python scripts/run.py assets/leakage-clean.json
The first synthetic table returns "status": "ISSUES_FOUND", "readiness": "REVIEW" with
"issues": ["EXACT_FEATURE_DUPLICATES", "CROSS_SPLIT_FEATURE_LEAKAGE", "CROSS_SPLIT_GROUP_LEAKAGE"]
and the offending row indexes ([0, 2], one in test, one in train). The second is a clean design and
returns NO_CHECKED_ISSUES for every supplied check while readiness stays REVIEW.
Preparing the input
Decide first what one observation is (a sample, a visit, a plate well), where predictions originate
in time, which column identifies the entity that must not straddle splits, and how the split was
designed. Then call szl_audit_dataset in kernel.py on explicitly bounded rows, or write the JSON
the CLI reads. Use content-defining feature columns that exclude row ids and split names. Add a
leakage_spec to check entity, temporal and preprocessing evidence; bind transform fit row ids to the
evidence you actually have, and distinguish when a label became available from when it was observed.
A TRAIN_ONLY declaration alone does not establish train-only fitting. Read references/contract.md
when preparing that manifest or interpreting missing evidence.
The helper never samples silently, never imputes missing timestamps, and never infers timezone or column meaning. Metadata records the source revision and license as declared without verifying them.
What it does not do
NO_CHECKED_ISSUES applies only to the supplied checks and selected rows; readiness remains REVIEW
until a human decides. Semantic near-duplicates, causal leakage through derived features, label
validity and population suitability are outside its reach and need separate investigation. Digests
are not anonymization. No network, model calls, training, upload or performance claim follows.
04 同仓库更多 Skills
szl-unit-invariants
Checks declared SI dimensions, allowlisted unit conversions, expected ranges and fixed pointwise invariants (equal, sum, difference, product, ratio) over selected measurement or simulation columns, flagging incompatible quantities, scale errors of 1000x and broken conservation checks. Use when merging instrument exports with different units, validating simulation output against conservation laws, or when a reviewer asks how unit errors were ruled out. Not an uncertainty-propagation or symbolic tool.
szl-uncertainty-lineage
Propagate declared measurement uncertainty through a small arithmetic dependency graph using a first-order delta approximation. Use when a researcher needs to see how corrected inputs or shared-source correlation change a reported standard uncertainty; require unit checks separately.
szl-typesafe-ai
TypeSafe/Jev: Choice/Noul/Score in code, not TypeScript types. System One models turn natural language and application state into probabilities that software can compose. Use when building routing, ranking, extraction, verification, or replacing an LLM prompt-and-parse step with structured decisions; also when brainstorming what TypeSafe could make possible. Do not use for TypeScript/Java/Rust static types, Zod/Pydantic/mypy/JSON Schema/protobuf codegen, or asking whether a pipeline is type-safe. At SZL, Jev is an optional second reader only.
demo
暂无简介
demo
暂无简介
szl-skill-update-review
Compare pinned local skill packages using inventories and a retained lock; report changed skills, files, declarations and reruns without approving safety or validity.