AGENT SKILL

model-training

Configure, preflight, run, resume, and troubleshoot DeepDanbooru 1.0.0 model training with a CPU-first TensorFlow workflow.

License 许可
MIT
最近更新
2026-09-03
Stars
★ 351
Forks
27

01 前言 Front Matter

name
model-training
license
MIT
metadata
{ "disco-role": "operating" }
description
Configure, preflight, run, resume, and troubleshoot DeepDanbooru 1.0.0 model training with a CPU-first TensorFlow workflow.
disable-model-invocation
true

02 资源文件 6

03 SKILL.md 正文

Model training

Use this sub-skill after a DeepDanbooru project and Danbooru-style dataset exist. It covers configuration, a read-only preflight, training dispatch, checkpoint resume, and saved-model artifacts for DeepDanbooru 1.0.0.

CPU TensorFlow is the required verified backend. GPU execution is optional and unverified; never infer GPU readiness from device discovery or a CPU success. Full training is intentionally skip-expensive during skill verification: the ResNet variants, image pipeline, multiple epochs, checkpoint I/O, and model exports can consume substantial compute, memory, and storage. Verify structure and a bounded data sample first, then launch training only with an explicit runtime and resource decision.

Route the request

Need Read or run
Create project.json, tags.txt, or the SQLite/image layout project-data-setup
Validate configuration values without training validate_project_config.py
Validate config, tags, SQLite rows, image paths, checkpoints, and packages training_preflight.py
Start, resume, or initialize from a saved model Training workflows
Understand row selection, image paths, labels, and augmentation Training data pipeline
Choose a model, optimizer, loss, schedule, or precision policy Model and loss reference
Diagnose empty runs, missing images, unsupported values, or resume surprises Troubleshooting
Evaluate the resulting Keras model inference-evaluation
Convert to TFLite or create Grad-CAM output post-training-tools

Supported dispatch values

Use these exact lowercase strings in project.json:

  • Models: resnet_152, resnet_custom_v1, resnet_custom_v2, resnet_custom_v3, resnet_custom_v4.
  • Optimizers: adam, sgd, rmsprop.
  • Losses: binary_crossentropy, focal_loss.

Any other value fails before training. The default project uses resnet_custom_v2, adam, binary_crossentropy, and an image shape of 299 by 299 with three channels. Shape order in the model is (image_height, image_width, 3) even though the JSON fields list width and height separately.

Safe preflight

From this sub-skill directory, run:

python scripts/validate_project_config.py PROJECT --check-paths
python scripts/training_preflight.py PROJECT --check-packages

Both helpers are read-only, deterministic, offline, and use the standard library for configuration, filesystem, tag, and SQLite checks. They do not launch training. Package and TensorFlow import probes are opt-in so basic validation still works when optional runtime packages are absent.

The preflight checks a bounded, ID-ordered image sample by default. Use --max-image-checks 0 only when a complete filesystem scan is intended and its I/O cost is acceptable. Use --probe-imports --probe-tensorflow to test the installed modules and confirm that TensorFlow exposes a CPU; this still does not train or verify a GPU.

Do not launch training unless all of these hold:

  1. project.json parses and all dispatch values and numeric ranges are valid.
  2. tags.txt is non-empty and is the intended fixed output vocabulary.
  3. The SQLite posts table has the required columns and at least one eligible row at minimum_tag_count.
  4. Derived image paths exist for the checked rows, and the image format is actually decodable by the training environment.
  5. Checkpoint state is understood: an existing latest checkpoint causes an automatic resume.
  6. The CPU TensorFlow environment can import DeepDanbooru, TensorFlow, and TensorFlow I/O.

Launch boundary

The side-effecting command is:

deepdanbooru train-project PROJECT_PATH

To load initial weights/model structure from an existing saved model:

deepdanbooru train-project PROJECT_PATH --source-model MODEL.keras

--source-model is not a no-resume flag. After the model is loaded, any latest checkpoint under PROJECT_PATH/checkpoints/ is restored and can supersede its state. Preserve or deliberately relocate checkpoints before changing training lineage. Also confirm source-model input/output compatibility with the image shape and number of tags; the training function does not perform a friendly compatibility check.

Runtime behavior to preserve

Training compiles with precision and recall metrics, shuffles records once per epoch from a checkpointed seed, applies an optional epoch learning-rate schedule, trains in checkpoint-sized slices, and saves after each slice. Checkpoint state includes optimizer/model variables plus used_epoch, used_minibatch, used_sample, offset, and random_seed; only the latest three checkpoints are retained.

Periodic exports are named:

model-MODEL.eEPOCH.keras

The final export is:

model-MODEL.keras

Exports omit optimizer state. Resume therefore depends on checkpoints/, not on a .keras export. export_model_per_epoch: 0 means export every epoch; otherwise an epoch export occurs when the completed epoch is divisible by that positive interval.

With mixed_precision: true, training uses a mixed_float16 model and a loss-scaled optimizer, then attempts separate float32 exports ending in .float32.keras. The 1.0.0 implementation reconstructs a float32 model and restores checkpointed model variables into it. Treat those exports as needing explicit load/inference validation. In particular, combining --source-model with mixed precision leaves the float32 export model unconstructed and can fail during export; the bundled preflight rejects that combination.

Failure semantics

A missing image is not reliably fatal at the dataset boundary because Dataset.ignore_errors() can drop load/decode failures. A run with many bad paths can therefore appear to progress while using fewer samples; a run with no usable records can produce checkpoints or model files without meaningful updates. Treat missing files, empty eligible selections, empty tag records, and zero usable batches as failures, not warnings to ignore.

Unsupported model, optimizer, or loss strings fail during dispatch. A checkpoint_frequency_mb of zero can prevent offset progress; a console_logging_frequency_mb of zero causes modulo-by-zero; non-positive batch/image/epoch values are invalid for an intended training run. Use the validators rather than discovering these conditions after allocating a long job.

Handoff

After training, retain project.json, the exact ordered tags.txt, the final .keras model, and any checkpoints needed for further resume. Validate the saved model through inference-evaluation before conversion or Grad-CAM work in post-training-tools. If project/data checks fail, return to project-data-setup instead of changing model hyperparameters to mask a data problem.

04 同仓库更多 Skills

VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-triton-opt

Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-solve

END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-skill

PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-pattern-fusion

Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-orchestrate

Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-feedback

Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).

★ 351