tokenizers
Router for VQ, VQGAN, Stable Diffusion VAE, and Consistency Decoder tokenizer training, finetuning, reconstruction, and round-trip checks.
- License 许可
- MIT
- 最近更新
- 2026-10-08
- Stars
- ★ 11
- Forks
- 3
01 前言 Front Matter
- name
- tokenizers
- license
- MIT
- metadata
- { "disco-role": "operating" }
- description
- Router for VQ, VQGAN, Stable Diffusion VAE, and Consistency Decoder tokenizer training, finetuning, reconstruction, and round-trip checks.
- disable-model-invocation
- true
02 资源文件 4
- 参考文档references/checkpoints.md2.1 KB
- 参考文档references/cli-reference.md3.4 KB
- 参考文档references/troubleshooting.md3.2 KB
- 参考文档references/workflows.md1.8 KB
03 SKILL.md 正文
Tokenizers
Use this sub-skill for image-tokenizer work that stays inside tokenizer and reconstruction behavior.
Owns
- VQ tokenizer training and finetuning.
- Reconstruction through VQ, VQGAN, Stable Diffusion VAE, and Consistency Decoder checkpoints.
- Image/code round-trip sanity checks for saved tokenizer codes.
- Legacy checkpoint migration notes for VQGAN
.ckptto.pthconversion.
Routes out
- Code extraction, dataset cache prep, and other preprocessing ->
data-preparation. - Class-conditional generation, sampling, serving, and evaluation ->
class-conditional. - Text-conditional generation and evaluation ->
text-conditional. - Checkpoint publishing and remote mutation -> excluded.
Best entry points
- Training:
scripts/train_vq.sh,scripts/train_vq_finetune.sh,scripts/train_vq_finetune_continue.sh - VQ reconstruction:
scripts/reconstruct_vq.sh - VQ validation / input sanity:
scripts/validate_vq.sh - VQGAN reconstruction:
scripts/reconstruct_vqgan.sh - Stable Diffusion VAE reconstruction:
scripts/reconstruct_vae.sh - Consistency Decoder reconstruction:
scripts/reconstruct_consistency_decoder.sh - Code/image round trip:
scripts/check_image_codes.py
Read before answering
references/workflows.mdreferences/cli-reference.mdreferences/checkpoints.mdreferences/troubleshooting.md
Fast routing rules
- If the request mentions code extraction, T5 extraction, or dataset cache layout, send it to
data-preparation. - If it asks for c2i or t2i generation, send it to the generation sub-skill instead of this one.
- If a checkpoint decode fails, first check the model family, codebook size, image size, and
pretrained_modelsor diffusers layout.
04 同仓库更多 Skills
obsidian-clipper-template-creator
Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.
obsidian-clipper-template-creator
Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.
python-api
Programmatic Object-Detection-Metrics API guidance for BoundingBox, BoundingBoxes, Evaluator, enums, coordinate conversion, AP/mAP semantics, and source-style import diagnostics.
file-evaluation
Folder-based PASCAL VOC AP/mAP evaluation from ground-truth and detection text files.
object-detection-metrics
Guides legacy Object-Detection-Metrics PASCAL VOC AP/mAP evaluation from detection text folders and direct Python API classes.
nx-workspace-patterns
Configure and optimize Nx monorepo workspaces. Use when setting up Nx, configuring project boundaries, optimizing build caching, or implementing affected commands.