cdf-anomalygen-release
Use when building the cdf-anomalygen Docker image (develop / product / air-gapped) from docker/Dockerfile — build the chosen target from source and run it. Not for training or synthetic defect-image generation (SDG).
- License 许可
- Apache-2.0
- 最近更新
- 2026-11-12
- Stars
- ★ 4
- Forks
- 0
01 前言 Front Matter
- name
- cdf-anomalygen-release
- license
- Apache-2.0
- metadata
- { "tags": [ "physical-ai", "docker", "release", "air-gapped" ], "owner": "NVIDIA", "author": "NVIDIA <nvidia@nvidia.com>", "service": "docker", "version": "1.2.0", "languages": [ "shell" ], "frameworks": [ "docker" ] }
- description
- Use when building the cdf-anomalygen Docker image (develop / product / air-gapped) from docker/Dockerfile — build the chosen target from source and run it. Not for training or synthetic defect-image generation (SDG).
- compatibility
- Requires Docker 23+ (BuildKit) + nvidia-container-toolkit and a CUDA GPU. The build compiles the heavy CUDA extensions from source and takes >1 h.
02 资源文件 3
- 参考文档BENCHMARK.md7.2 KB
- 参考文档references/running.md1.7 KB
- 参考文档skill-card.md4.1 KB
03 SKILL.md 正文
Skill: cdf-anomalygen-release
Purpose
Build and run the cdf-anomalygen container. docker/Dockerfile reproduces
scripts/env_setup.sh as a two-stage image (compile heavy wheels → slim non-root runtime).
This is a release/build workflow — not training or inference
(use the cdf-anomalygen skill for those). Build targets, GPU-arch lists, and troubleshooting
detail live in docker/README.md. Run every command from the repo root.
Check whether a build is needed at all. Prebuilt images are published on NGC:
docker pull nvcr.io/nvidia/cdf-anomalygen:1.2.0needs no login and no compile. Build only when the user wants a modified image, a target NGC does not publish, or an air-gapped bundle. A build from source takes >1 h — say so before starting one.
When to Use This Skill
Use this skill when the user wants to build or ship the anomalygen container — build a develop / product / air-gapped image, push it to a registry, or run it. E.g. "build the anomalygen product container", "build an air-gapped develop image".
Do not use it to run training or SDG generation — that's the cdf-anomalygen skill.
Prerequisites
- Docker 23+ (BuildKit on by default) + nvidia-container-toolkit + a CUDA GPU.
- Enough headroom for the source compile: the heavy extensions are memory-hungry and the build needs tens of GB of disk for intermediate layers.
- The repo
.venv(scripts/env_setup.sh: py3.13, torch 2.13.0+cu132) is optional — it is used only to auto-deriveTORCH_CUDA_ARCH_LISTbelow, viatorch.cuda.get_arch_list(). Without it the documented default is used. - Air-gapped only: the model checkpoints under
./checkpoints/.scripts/download_checkpoints.shpopulates this directory.
Targets
docker/Dockerfile exposes four targets (--target, default develop):
| Target | What |
|---|---|
develop (default) |
non-root interactive image + dev tooling (pytest, ruff, pre-commit) |
product |
non-root, app code read-only, ANOMALYGEN_PRODUCT_MODE=1, no dev tooling |
airgapped-develop |
develop + the checkpoints baked in for offline runs |
airgapped-product |
product + the checkpoints baked in for offline runs |
Instructions
Default flow: Step 1, then stop with the built image. Push (Step 2) and run (Step 3) are opt-in — only do them when the user explicitly asks to publish or run the image.
Set the shared variables once (arm64 host → use the commented values):
export TARGET=develop # develop | product | airgapped-develop | airgapped-product
export IMAGE=cdf-anomalygen:cuda-13.2.1-${TARGET}-ubuntu24.04-amd64
# tag = <base-cuda>-<target>-<ubuntu>-<arch>; arm64 host → …-arm64. Set TARGET *first*: it feeds both
# the tag and --target below, so they cannot disagree. Hardcoding "develop" here and then building
# --target product silently ships a product image labelled develop.
export PLATFORM=linux/amd64 # arm64 host: linux/arm64
# GPU arch (>=8.0), derived from .venv torch when present; forwarded to the build as a --build-arg.
# The `:-` matters: get_arch_list() is [] with no GPU (and the command fails outright with no .venv),
# and a bare --build-arg forwards that empty value, overriding the Dockerfile default.
export TORCH_CUDA_ARCH_LIST="$(.venv/bin/python -c "import torch; print(' '.join(c for c in sorted({f'{a[3:-1]}.{a[-1]}' for a in torch.cuda.get_arch_list() if a.startswith('sm_')}, key=float) if float(c) >= 8.0))" 2>/dev/null)"
export TORCH_CUDA_ARCH_LIST="${TORCH_CUDA_ARCH_LIST:-8.0 8.6 9.0 10.0 12.0}" # arm64: '9.0 10.0 10.3 12.0'
Step 1 — build the image
Stage 1 compiles flash-attn / transformer-engine / apex / flash-attn-3-nv / natten from
source. This takes >1 h — it is not a hang. TORCH_CUDA_ARCH_LIST is required on
this path: docker build never sees a GPU, so nothing can be autodetected.
Standard (develop / product):
docker build --platform "$PLATFORM" -f docker/Dockerfile \
--build-arg TORCH_CUDA_ARCH_LIST \
--target "$TARGET" -t "$IMAGE" . # TARGET from the shared vars; tag and target stay in sync
Air-gapped (airgapped-develop / airgapped-product) — needs buildx + a named
ckpts context (which bypasses .dockerignore). The image is self-contained, so save it
to ./results/ as a gzipped bundle for offline transfer and report that path to the user:
# --load is required: buildx's default driver keeps the result in its own build cache, so without it
# the tag never reaches the local daemon and the `docker save` below fails with "reference does not
# exist". (Single-platform builds only — --load cannot export a multi-arch manifest.)
docker buildx build --platform "$PLATFORM" -f docker/Dockerfile \
--build-arg TORCH_CUDA_ARCH_LIST \
--build-context ckpts=./checkpoints \
--target "$TARGET" -t "$IMAGE" --load . # set TARGET=airgapped-develop|airgapped-product first
mkdir -p results
BUNDLE="results/$(basename "$IMAGE" | tr ':' '_').tar.gz"
docker save "$IMAGE" | gzip > "$BUNDLE"
echo "air-gapped bundle: $BUNDLE" # tell the user this location
Step 2 — push (only when the user asks to publish; skip by default)
Ask the user which registry to publish to — there is no default. Tag the local image for that registry, authenticate, then push:
export REGISTRY=<your-registry> # e.g. registry.example.com or an org path
docker tag "$IMAGE" "$REGISTRY/$IMAGE"
docker login "$REGISTRY" # credentials per your registry; --password-stdin in CI
docker push "$REGISTRY/$IMAGE"
Step 3 — run (only when the user asks to run the container; skip by default)
docker run invocation, the --user / -e USER pairing that makes getpass.getuser() resolve,
--shm-size, HF_TOKEN, and loading an air-gapped bundle:
running.md.
Verification
Run these after Step 1 has produced an image — they check that image, and the smoke test needs the
Step 3 docker run invocation, so they are not a standalone entry point.
-
Image exists:
docker images "$IMAGE". -
product only — guardrails hold:
ANOMALYGEN_PRODUCT_MODE=1is set, it runs as the non-rootnvidiauser, and the app code is read-only. If any fails, the image is not productized. -
Smoke test — in the Step 3 container (repo +
checkpoints/mounted; baked in for air-gapped), a 20-iteration dry run finishes without error.The output root must be fresh, or this check is worthless. A reused root resumes a leftover checkpoint and finishes instantly, which looks identical to success. The
date +%ssuffix below is what makes it fresh — keep it, and confirm the run actually logs 20 iterations.
IMAGINAIRE_OUTPUT_ROOT="$PWD/results/dryrun-$(date +%s)" torchrun --nproc_per_node=1 \
anomalygen/scripts/texture/train.py --config=cosmos_framework/configs/base/config.py \
--recipe=ag_config/exp_texture_ft_phone_screen.yaml \
-- experiment=anomalygen_texture_ft trainer.max_iter=20 trainer.validation_iter=10 checkpoint.save_iter=10
Examples
"Build the anomalygen product container" — shared variables, then Step 1 with the product target; stop there (no push unless asked):
export TARGET=product # re-export IMAGE so the tag follows the target
export IMAGE=cdf-anomalygen:cuda-13.2.1-${TARGET}-ubuntu24.04-amd64
docker build --platform "$PLATFORM" -f docker/Dockerfile \
--build-arg TORCH_CUDA_ARCH_LIST \
--target "$TARGET" -t "$IMAGE" .
"Build an air-gapped develop image" — Step 1's buildx variant with the ckpts named context,
then load the saved bundle on the offline host. The checkpoints must already be under
./checkpoints/.
Troubleshooting
- "TORCH_CUDA_ARCH_LIST is empty and could not be derived" → something overrode the Dockerfile
default with an empty value: a bare
--build-argforwards an empty env var, andget_arch_list()is[]with no GPU. The shared vars guard this with:-; otherwise set it by hand (amd648.0 8.6 9.0 10.0 12.0, arm649.0 10.0 10.3 12.0). - Build seems stuck for an hour → that is the source compile of flash-attn / TE / apex, which is expected. Don't kill it; watch the layer output instead.
- Build OOM (flash-attn / TE / apex) → lower the job count with
--build-arg MAX_JOBS=…(default 16). - Missing
--shm-size(min16g) → "Bus error" / silent hangs: the dataloader uses/dev/shmand Docker's 64 MB default is too small. Always pass it. failed to find stage "…"→ bad--target; valid:develop,product,airgapped-develop,airgapped-product.- Don't build inside a product runtime (
ANOMALYGEN_PRODUCT_MODE=1) — build from a clean clone or a develop container.
04 同仓库更多 Skills
agentic-ecology-ui
Guidelines and templates for creating a generic web UI to visualize, rank, filter, and annotate vector databases. The UI has a hierarchical structure covering database selection, label selection/definition, ranking/filtering configuration, and row visualization/annotation. Use when building, serving, or adapting an interactive web UI for Hoplite vector databases.
agentic-ecology-init
Initializes a local uv-managed project directory for agentic ecology workloads. Sets up Python dependencies using reference pyproject.toml and uv.lock, configures workspace rules, and ensures Agentic Ecology skills are discoverable. Use when initializing a new project workspace or bootstrapping a clean environment.
agentic-ecology-camera-traps
Provides capabilities to run SpeciesNet detector and classifier on camera trap images, extract crop-level feature embeddings, and populate a Hoplite vector database for downstream search and agile modeling. Use when ingesting camera trap images, classifying wildlife photos, or indexing camera trap data.
agentic-ecology-bioacoustics
Provides bioacoustic analysis capabilities for ecologists and researchers using the perch-hoplite Python package. A typical use case is to use agile modeling to bootstrap the creation and deployment of a bespoke detector for targeted species on an existing collection of passive acoustic monitoring recordings. Use when processing and annotating audio recordings for bioacoustic applications.
sample-skill
Sample skill used by agent-env integration tests. Exercises SKILL.md parsing plus supporting scripts and references.
simready-foundation-validate-foundation-change
Use for auditing SimReady requirement, validator, feature, profile, adapter, test, and skill consistency.