llama-factory
Use for LlamaFactory repository workflows: training configs, dataset/template preparation, model loading/export, inference/API/Web UI operations, and experimental USE_V1 flows.
- License 许可
- Apache-2.0
- 最近更新
- 2026-09-03
- Stars
- ★ 351
- Forks
- 27
01 前言 Front Matter
- name
- llama-factory
- license
- Apache 2.0
- metadata
- { "disco-role": "operating" }
- description
- Use for LlamaFactory repository workflows: training configs, dataset/template preparation, model loading/export, inference/API/Web UI operations, and experimental USE_V1 flows.
- disable-model-invocation
- true
02 资源文件 3
- 参考文档references/repo-provenance.md1.5 KB
- 参考文档references/repo-routing-metadata.json567 B
- 参考文档references/troubleshooting.md4.1 KB
03 SKILL.md 正文
LlamaFactory
Use this skill when a user asks for help with LlamaFactory (llamafactory package, llamafactory-cli, or lmf) rather than generic Hugging Face training. The default architecture is v0; route to v1-experimental only when the user explicitly sets or asks about USE_V1=1 or v1-style configs.
Start Here
- Identify the requested surface: data/template, training config, model/export, inference/API, Web UI/ops, or v1.
- If the task involves a runnable command, prefer static validation first: inspect config keys, render the intended command, or run a bundled helper with
--helpbefore starting training, downloads, API servers, or distributed launches. - Confirm optional dependencies and hardware before recommending vLLM, SGLang, DeepSpeed, FlashAttention, bitsandbytes, GPTQ, AWQ, HQQ, EETQ, Unsloth, Liger Kernel, KTransformers, NPU, Ray, FSDP, or Megatron-core paths.
- Keep credentials and local machine paths out of reusable examples: hub tokens, tracker keys, private model paths, and dataset paths should be placeholders.
- Read
references/troubleshooting.mdfor cross-cutting install/import, dependency, CLI, data/config, backend, and environment-variable issues.
Route By Task
| User request | Use |
|---|---|
Registering datasets, fixing dataset_info.json, Alpaca/ShareGPT/OpenAI rows, multimodal columns, tool-calling rows, templates, preprocessing, packed data |
sub-skills/data-and-templates/ |
Writing or debugging llamafactory-cli train YAML/JSON, CLI overrides, LoRA/QLoRA/full/freeze, SFT/PT/RM/PPO/DPO/KTO, distributed launch, logging |
sub-skills/training-and-configs/ |
Using ChatModel, chat, webchat, api, OpenAI-compatible endpoints, streaming, scores, infer_backend, vLLM or SGLang inference |
sub-skills/inference-and-serving/ |
| Loading tokenizers/models/processors, adapter setup, LoRA merge/export, quantization, checkpoint conversion/init utilities, hub mirror behavior | sub-skills/model-loading-and-export/ |
Installing/running LlamaFactory, help, version, env, LlamaBoard webui, operational env vars, monitors, Docker/backend notes |
sub-skills/webui-and-ops/ |
Experimental v1 trainer/core/plugin/config flows selected with USE_V1=1 |
sub-skills/v1-experimental/ |
CLI And Package Facts
- Package/distribution:
llamafactory. - Public version in this repo snapshot:
0.9.6.dev0. - Console scripts:
llamafactory-cliandlmf, both dispatch tollamafactory.cli:main. - v0 CLI routes include
train,chat,api,export,webchat,webui,env,version, andhelp. - Multi-GPU training may be wrapped in
torchrunautomatically when multiple devices are visible, or forced withFORCE_TORCHRUN=1. USE_V1=1switches the launcher to the experimental v1 architecture and changes supported commands/config shape.
Minimal Checks
Use these checks after installing LlamaFactory in a real environment:
python - <<'PY'
import llamafactory
print(llamafactory.__version__)
PY
llamafactory-cli help
If the package is only partially installed, root imports may succeed while workflow imports fail later. Run the sanity helper in sub-skills/webui-and-ops/scripts/llamafactory_sanity_check.py to distinguish metadata/import/CLI/dependency problems.
Bundled Helpers
sub-skills/data-and-templates/scripts/validate_dataset_entry.pychecks dataset registry entries and tiny row files without importing LlamaFactory.sub-skills/training-and-configs/scripts/render_train_command.pymerges simple YAML/JSON overrides and prints a safe training command.sub-skills/training-and-configs/scripts/eval_bleu_rouge.pyprovides dependency-light prediction metric checks for generated JSONL outputs.sub-skills/inference-and-serving/scripts/openai_api_smoke.pyprobes an OpenAI-compatible local API when the user supplies server details.sub-skills/inference-and-serving/scripts/vllm_batch_infer.pywraps batch inference intent and requires the caller to provide an installed inference environment.sub-skills/model-loading-and-export/scripts/check_export_config.pyflags static export/merge/quantization config pitfalls.sub-skills/webui-and-ops/scripts/llamafactory_sanity_check.pychecks Python/package/CLI availability and common optional modules.sub-skills/v1-experimental/scripts/check_v1_config_keys.pywarns about mixed v0/v1 config keys without importing LlamaFactory.
Repo-Level References
references/repo-provenance.mdrecords the source snapshot and evidence paths used to build this skill.references/troubleshooting.mdcovers cross-cutting failures and points to the nearest owning sub-skill.
Safety Notes
- Do not run model downloads, training, distributed launches, API servers, Web UI sessions, checkpoint conversion, or export jobs unless the user asks and the environment/hardware are suitable.
- Treat repo examples and tests as evidence, not runtime dependencies for this skill. The bundled references and scripts are the self-contained runtime material.
- For dependency-heavy native verification, use a full LlamaFactory installation with PyTorch and the selected optional backend packages.
04 同仓库更多 Skills
passnet-triton-opt
Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.
passnet-solve
END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.
passnet-skill
PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.
passnet-pattern-fusion
Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.
passnet-orchestrate
Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.
passnet-feedback
Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).