dalle2-pytorch
Use the DALLE2-pytorch package for DALL-E 2 style text-to-image model APIs, diffusion prior/decoder training, WebDataset data layouts, and tracker/checkpoint workflows.
- License 许可
- MIT
- 最近更新
- 2026-10-08
- Stars
- ★ 11
- Forks
- 3
01 前言 Front Matter
- name
- dalle2-pytorch
- license
- MIT
- metadata
- { "disco-role": "operating" }
- description
- Use the DALLE2-pytorch package for DALL-E 2 style text-to-image model APIs, diffusion prior/decoder training, WebDataset data layouts, and tracker/checkpoint workflows.
- disable-model-invocation
- true
02 资源文件 3
- 参考文档references/repo-provenance.md2.2 KB
- 参考文档references/repo-routing-metadata.json421 B
- 参考文档references/troubleshooting.md3.1 KB
03 SKILL.md 正文
DALLE2-pytorch
Use this repo skill when a task involves the dalle2-pytorch package, DALL-E 2 style CLIP-latent text-to-image generation, diffusion prior training, cascaded decoder training, DALLE2 checkpoints, WebDataset embedding data, or the package's training/experiment tracking utilities.
What This Skill Covers
- Public Python APIs for
DALLE2,DiffusionPriorNetwork,DiffusionPrior,Unet,Decoder, CLIP adapters, VQGAN/VAE latent diffusion, inpainting, tokenization, and thedreamCLI. - JSON config classes and bundled training launchers for decoder and diffusion-prior training.
- WebDataset and EmbeddingReader data layouts, sidecar embedding folders, shard/index validation, trackers, checkpoint loading, and checkpoint saving.
- Troubleshooting for install/import, CLIP downloads, CPU vs GPU expectations, config validation, data layout, tracker credentials, and checkpoint format mismatches.
Install And Minimal Check
Install the public package:
python -m pip install dalle2-pytorch
Run the bundled package check from this skill directory:
python scripts/check_install.py --mode imports
python scripts/check_install.py --mode cli-help
For a CPU-only synthetic API smoke test that avoids CLIP downloads and model weights:
python sub-skills/generation-and-api/scripts/check_dalle2_runtime.py --mode tiny-forward
Choose A Sub-Skill
generation-and-api
Read sub-skills/generation-and-api/SKILL.md for Python model construction and generation tasks:
- Build or call
DALLE2,DiffusionPrior,Decoder,Unet, CLIP adapters, orVQGanVAE. - Use
dream, load generation checkpoints, tokenize prompts, sample images, or perform decoder inpainting. - Debug checkpoint architecture mismatches, CLIP weight downloads, CPU/GPU limitations, inpaint mask shape, or classifier-free guidance.
training-and-configs
Read sub-skills/training-and-configs/SKILL.md for training and configuration tasks:
- Author/validate
TrainDecoderConfigorTrainDiffusionPriorConfigJSON. - Use
DecoderTrainerorDiffusionPriorTrainerdirectly. - Build
pythonoraccelerate launchcommands, run bundled training wrappers, resume/save trainer checkpoints, or debugunet_training_mask, resampling, DeepSpeed fp16, and config errors.
data-and-tracking
Read sub-skills/data-and-tracking/SKILL.md for data and experiment-management tasks:
- Validate decoder WebDataset shard names, sample keys, sidecar image/text embedding folders,
shard_width, orindex_width. - Use
ImageEmbeddingDataset,create_image_embedding_dataloader,get_reader,make_splits, orPriorEmbeddingDataset. - Configure console/W&B logging, local/URL/W&B checkpoint loading, or local/W&B/HuggingFace saving.
Important Constraints
- Real training and useful image sampling normally need GPU memory and trained checkpoints. CPU is suitable for imports, config parsing, CLI help, and tiny synthetic forward-loss tests.
- CLIP adapters (
OpenAIClipAdapter,OpenClipAdapter) may download weights and require network/cache. Do not instantiate them in offline smoke checks unless weights are already cached. - Training examples may need datasets, W&B/HuggingFace/S3 credentials, metric assets, and long runtime. Validate configs and data layout before launch.
- The
dreamCLI expects a combined DALLE2 checkpoint with package-expected init/model keys; trainer checkpoints are a different format. - If
clip-anytorchfails becausepkg_resourcesis missing, installsetuptools<81and rerunpython -m pip check.
Root References
- Repository snapshot and refresh baseline: references/repo-provenance.md.
- Cross-cutting install/import/backend troubleshooting: references/troubleshooting.md.
- Router metadata used by managed import: references/repo-routing-metadata.json.
Self-Containment Rule
Use the bundled references and scripts in this skill tree. Do not require future agents to open, run, or copy scripts, docs, configs, tests, or examples from an original DALLE2-pytorch checkout. When a user wants to train in a checkout intentionally, treat that checkout as their working project, not as this skill's required evidence source.
04 同仓库更多 Skills
obsidian-clipper-template-creator
Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.
obsidian-clipper-template-creator
Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.
python-api
Programmatic Object-Detection-Metrics API guidance for BoundingBox, BoundingBoxes, Evaluator, enums, coordinate conversion, AP/mAP semantics, and source-style import diagnostics.
file-evaluation
Folder-based PASCAL VOC AP/mAP evaluation from ground-truth and detection text files.
object-detection-metrics
Guides legacy Object-Detection-Metrics PASCAL VOC AP/mAP evaluation from detection text folders and direct Python API classes.
nx-workspace-patterns
Configure and optimize Nx monorepo workspaces. Use when setting up Nx, configuring project boundaries, optimizing build caching, or implementing affected commands.