[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M78T4F4ENVT56AV6KBKT5PF2":3,"skill-same:01M78T4F4ENVT56AV6KBKT5PF2":67},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":27,"commitSha":28,"skillDirectory":29,"forksCount":30,"repositoryHtmlUrl":31,"repositoryHomepageUrl":6,"ownerHtmlUrl":32,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":33,"install":34,"resources":46},"01M78T4F4ENVT56AV6KBKT5PF2","sft-and-pretraining",null,"Guides agents creating Axolotl SFT and continual pretraining configs, LoRA or QLoRA recipes, preprocessing workflows, checkpoint resumes, sample packing, and stability triage.","VectorSpaceLab\u002FAREX-Skill","VectorSpaceLab","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F180637102?v=4","01M7538C5WPSJGWF7KHR4EFZ9F","01M7538C6R2GFP53NPXTCZNBRT",351,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Faxolotl\u002Fsub-skills\u002Fsft-and-pretraining\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: sft-and-pretraining\ndescription: \"Guides agents creating Axolotl SFT and continual pretraining\n  configs, LoRA or QLoRA recipes, preprocessing workflows, checkpoint resumes,\n  sample packing, and stability triage.\"\ndisable-model-invocation: true\nmetadata:\n  disco-role: operating\nlicense: Apache 2.0\n---\n\n# SFT and Pretraining\n\nUse this sub-skill when the task is to fine-tune with Axolotl using supervised fine-tuning, adapt a pretraining recipe, create a LoRA or QLoRA starter config, debug SFT\u002Fpretraining loss behavior, reason about checkpoint resume fields, or decide whether to run preprocessing before training.\n\nAxolotl is config-driven: produce or edit a YAML config first, validate it, then use the same config with `axolotl preprocess`, `axolotl train`, `axolotl inference`, and `axolotl merge-lora` as appropriate.\n\n## Read First\n\n- [references\u002Fworkflows.md](references\u002Fworkflows.md) for SFT, completion-style pretraining, streaming pretraining, validation order, and checkpoint\u002Fresume workflow.\n- [references\u002Ftraining-options.md](references\u002Ftraining-options.md) for LoRA\u002FQLoRA\u002Ffull fine-tune choices, core config fields, packing, batch sizing, output fields, and preprocessing reuse.\n- [references\u002Ftroubleshooting.md](references\u002Ftroubleshooting.md) for NaN\u002Floss spikes, OOM, tokenizer mismatch, packing length issues, dataset-prepared reuse, checkpoint confusion, and network\u002Fmodel path problems.\n- [scripts\u002Fmake_minimal_sft_config.py](scripts\u002Fmake_minimal_sft_config.py) for a safe local helper that emits a starter SFT YAML without downloading models, loading datasets, or starting training.\n\n## Route Boundaries\n\n- Use this sub-skill for default SFT, chat-template\u002Finput-output\u002Fcompletion training flow, continual pretraining, minimal config construction, `axolotl preprocess` before expensive runs, `output_dir`, checkpoints, resume, `dataset_prepared_path`, and SFT\u002Fpretraining stability triage.\n- Route dataset schema details, custom field mappings, and prompt-format mechanics to `data-and-configs`.\n- Route DPO, IPO, KTO, ORPO, SimPO, reward modeling, GRPO, and EBFT to `preference-tuning` or `rl-and-rewards`.\n- Route model architecture quirks, tokenizer\u002Fmodel loading internals, LoRA target-module internals, and adapter compatibility to `model-loading-and-adapters`.\n- Route DeepSpeed\u002FFSDP launch topology, throughput tuning, vLLM serving, and distributed performance to `distributed-and-performance`.\n- Route complete CLI catalog, installation, fetching examples, inference UI, and operational command details to `cli-and-operations`.\n\n## Quick Workflow\n\n1. Pick the workflow: SFT via `datasets`, non-streaming pretraining via `datasets: [{type: completion}]`, or streaming pretraining via `pretraining_dataset`.\n2. Choose the training mode: LoRA for a common adapter baseline, QLoRA for 4-bit memory pressure, or full fine-tune when all weights should update and hardware allows it.\n3. Create a config with `base_model`, dataset source\u002Ftype, `output_dir`, sequence\u002Fbatch settings, optimizer\u002FLR, precision, checkpoint fields, and adapter fields when needed.\n4. Run `axolotl preprocess \u003Cconfig.yaml>` for non-streaming data when you want to validate tokenization, label masking, or reusable `dataset_prepared_path` before training.\n5. Run `axolotl preprocess \u003Cconfig.yaml> --debug` when label masking, chat template behavior, or empty\u002Foverlong examples are suspected.\n6. Only after validation, run `axolotl train \u003Cconfig.yaml>` in the user’s prepared Axolotl environment.\n\nFor a starter YAML, use:\n\n```bash\npython sub-skills\u002Fsft-and-pretraining\u002Fscripts\u002Fmake_minimal_sft_config.py \\\n  --base-model NousResearch\u002FLlama-3.2-1B \\\n  --dataset-path .\u002Fchat.jsonl \\\n  --dataset-type chat_template \\\n  --adapter qlora \\\n  --output-dir .\u002Foutputs\u002Fchat-qlora\n```\n\nThe helper only writes YAML to standard output or to a requested file; it does not inspect models, access the network, load datasets, or train.\n","# SFT and Pretraining\n\nUse this sub-skill when the task is to fine-tune with Axolotl using supervised fine-tuning, adapt a pretraining recipe, create a LoRA or QLoRA starter config, debug SFT\u002Fpretraining loss behavior, reason about checkpoint resume fields, or decide whether to run preprocessing before training.\n\nAxolotl is config-driven: produce or edit a YAML config first, validate it, then use the same config with `axolotl preprocess`, `axolotl train`, `axolotl inference`, and `axolotl merge-lora` as appropriate.\n\n## Read First\n\n- [references\u002Fworkflows.md](references\u002Fworkflows.md) for SFT, completion-style pretraining, streaming pretraining, validation order, and checkpoint\u002Fresume workflow.\n- [references\u002Ftraining-options.md](references\u002Ftraining-options.md) for LoRA\u002FQLoRA\u002Ffull fine-tune choices, core config fields, packing, batch sizing, output fields, and preprocessing reuse.\n- [references\u002Ftroubleshooting.md](references\u002Ftroubleshooting.md) for NaN\u002Floss spikes, OOM, tokenizer mismatch, packing length issues, dataset-prepared reuse, checkpoint confusion, and network\u002Fmodel path problems.\n- [scripts\u002Fmake_minimal_sft_config.py](scripts\u002Fmake_minimal_sft_config.py) for a safe local helper that emits a starter SFT YAML without downloading models, loading datasets, or starting training.\n\n## Route Boundaries\n\n- Use this sub-skill for default SFT, chat-template\u002Finput-output\u002Fcompletion training flow, continual pretraining, minimal config construction, `axolotl preprocess` before expensive runs, `output_dir`, checkpoints, resume, `dataset_prepared_path`, and SFT\u002Fpretraining stability triage.\n- Route dataset schema details, custom field mappings, and prompt-format mechanics to `data-and-configs`.\n- Route DPO, IPO, KTO, ORPO, SimPO, reward modeling, GRPO, and EBFT to `preference-tuning` or `rl-and-rewards`.\n- Route model architecture quirks, tokenizer\u002Fmodel loading internals, LoRA target-module internals, and adapter compatibility to `model-loading-and-adapters`.\n- Route DeepSpeed\u002FFSDP launch topology, throughput tuning, vLLM serving, and distributed performance to `distributed-and-performance`.\n- Route complete CLI catalog, installation, fetching examples, inference UI, and operational command details to `cli-and-operations`.\n\n## Quick Workflow\n\n1. Pick the workflow: SFT via `datasets`, non-streaming pretraining via `datasets: [{type: completion}]`, or streaming pretraining via `pretraining_dataset`.\n2. Choose the training mode: LoRA for a common adapter baseline, QLoRA for 4-bit memory pressure, or full fine-tune when all weights should update and hardware allows it.\n3. Create a config with `base_model`, dataset source\u002Ftype, `output_dir`, sequence\u002Fbatch settings, optimizer\u002FLR, precision, checkpoint fields, and adapter fields when needed.\n4. Run `axolotl preprocess \u003Cconfig.yaml>` for non-streaming data when you want to validate tokenization, label masking, or reusable `dataset_prepared_path` before training.\n5. Run `axolotl preprocess \u003Cconfig.yaml> --debug` when label masking, chat template behavior, or empty\u002Foverlong examples are suspected.\n6. Only after validation, run `axolotl train \u003Cconfig.yaml>` in the user’s prepared Axolotl environment.\n\nFor a starter YAML, use:\n\n```bash\npython sub-skills\u002Fsft-and-pretraining\u002Fscripts\u002Fmake_minimal_sft_config.py \\\n  --base-model NousResearch\u002FLlama-3.2-1B \\\n  --dataset-path .\u002Fchat.jsonl \\\n  --dataset-type chat_template \\\n  --adapter qlora \\\n  --output-dir .\u002Foutputs\u002Fchat-qlora\n```\n\nThe helper only writes YAML to standard output or to a requested file; it does not inspect models, access the network, load datasets, or train.\n",{"name":5,"license":23,"metadata":24,"description":7,"disable-model-invocation":26},"Apache 2.0",{"disco-role":25},"operating",true,"main","ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6","skills\u002Frepositories\u002Frepo-skills\u002Faxolotl\u002Fsub-skills\u002Fsft-and-pretraining",27,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab","2026-09-03 10:25:21",{"source":8,"skillName":5,"commands":35,"prompt":45,"zipAvailable":26},[36,39,42],{"tool":37,"command":38},"npx","npx skills add VectorSpaceLab\u002FAREX-Skill --skill sft-and-pretraining",{"tool":40,"command":41},"bunx","bunx skills add VectorSpaceLab\u002FAREX-Skill --skill sft-and-pretraining",{"tool":43,"command":44},"pnpm","pnpm dlx skills add VectorSpaceLab\u002FAREX-Skill --skill sft-and-pretraining","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Faxolotl\u002Fsub-skills\u002Fsft-and-pretraining\u002FSKILL.md",[47,53,57,61],{"path":48,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":51,"sourceHtmlUrl":52},"references\u002Ftraining-options.md","REFERENCE","参考文档",6195,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Faxolotl\u002Fsub-skills\u002Fsft-and-pretraining\u002Freferences\u002Ftraining-options.md",{"path":54,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":55,"sourceHtmlUrl":56},"references\u002Ftroubleshooting.md",7776,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Faxolotl\u002Fsub-skills\u002Fsft-and-pretraining\u002Freferences\u002Ftroubleshooting.md",{"path":58,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":59,"sourceHtmlUrl":60},"references\u002Fworkflows.md",6215,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Faxolotl\u002Fsub-skills\u002Fsft-and-pretraining\u002Freferences\u002Fworkflows.md",{"path":62,"resourceType":63,"resourceTypeDesc":64,"sizeBytes":65,"sourceHtmlUrl":66},"scripts\u002Fmake_minimal_sft_config.py","SCRIPT","脚本",8548,"https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Frepositories\u002Frepo-skills\u002Faxolotl\u002Fsub-skills\u002Fsft-and-pretraining\u002Fscripts\u002Fmake_minimal_sft_config.py",[68,75,82,89,96,103],{"skillNo":69,"name":70,"nameZh":6,"description":71,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":72,"categories":73,"keywords":74,"publishedAt":18},"01M799NYSH1R3RVAHSM2D88Y84","passnet-triton-opt","Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block\u002Fgrid\u002Fwarp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-triton-opt\u002FSKILL.md",[],[],{"skillNo":76,"name":77,"nameZh":6,"description":78,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":79,"categories":80,"keywords":81,"publishedAt":18},"01M799NS5APEC0N1FZ7XZVW6BN","passnet-solve","END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-solve\u002FSKILL.md",[],[],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":86,"categories":87,"keywords":88,"publishedAt":18},"01M799NNY9BKSM21V5R7QVTBJX","passnet-skill","PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under .\u002Fpass_dir\u002F, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-skill\u002FSKILL.md",[],[],{"skillNo":90,"name":91,"nameZh":6,"description":92,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":93,"categories":94,"keywords":95,"publishedAt":18},"01M799NHH6NP7Z1RK77KGXAN6S","passnet-pattern-fusion","Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()\u002Freplacement_args()\u002Freplacement_func() files, when a pass \"failed to match\", or when deciding how to fuse multiple kernels\u002Fops.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-pattern-fusion\u002FSKILL.md",[],[],{"skillNo":97,"name":98,"nameZh":6,"description":99,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":100,"categories":101,"keywords":102,"publishedAt":18},"01M799ND2J4WNC2PGAZ0A06NDF","passnet-orchestrate","Round\u002Fbatch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep\u002Frevert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-orchestrate\u002FSKILL.md",[],[],{"skillNo":104,"name":105,"nameZh":6,"description":106,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":18},"01M799N8SBTMY02ZDG2AZF5ZWE","passnet-feedback","Fast, deterministic feedback for PassNet work: pre-flight pattern\u002Fmatch verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).","https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAREX-Skill\u002Fblob\u002Fmain\u002Fskills\u002Ftask-oriented\u002FPassNet\u002Fskills\u002Fpassnet-feedback\u002FSKILL.md",[],[]]