finetuning
Fine-tune models on Azure AI Foundry using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset preparation, training job submission, deployment, and evaluation. USE FOR: fine-tune, SFT, DPO, RFT, training data, grader, distillation, fine-tuned model, training job, large file upload, calibrate grader, deploy fine-tuned model, evaluate fine-tuned model. DO NOT USE FOR: general model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).
- License 许可
- MIT
- 最近更新
- 2026-11-15
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- finetuning
- license
- MIT
- metadata
- { "author": "Microsoft", "version": "0.0.0-placeholder" }
- description
- Fine-tune models on Azure AI Foundry using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset preparation, training job submission, deployment, and evaluation. USE FOR: fine-tune, SFT, DPO, RFT, training data, grader, distillation, fine-tuned model, training job, large file upload, calibrate grader, deploy fine-tuned model, evaluate fine-tuned model. DO NOT USE FOR: general model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).
02 资源文件 33
- 参考文档references/agentic-rft.md3.2 KB
- 参考文档references/dataset-formats.md3.7 KB
- 参考文档references/deployment.md3.4 KB
- 参考文档references/evaluation.md5.8 KB
- 参考文档references/grader-design.md3.4 KB
- 参考文档references/hyperparameters.md3.2 KB
- 参考文档references/large-file-uploads.md927 B
- 参考文档references/platform-gotchas.md2.0 KB
- 参考文档references/reward-hacking.md2.9 KB
- 参考文档references/training-curves.md4.1 KB
- 参考文档references/training-types.md3.1 KB
- 参考文档references/vision-fine-tuning.md4.1 KB
- 脚本scripts/calibrate_grader.py9.3 KB
- 脚本scripts/check_training.py7.8 KB
- 脚本scripts/cleanup.py9.5 KB
- 脚本scripts/common.py7.6 KB
- 脚本scripts/convert_dataset.py10.7 KB
- 脚本scripts/deploy_model.py9.4 KB
- 脚本scripts/evaluate_model.py11.4 KB
- 脚本scripts/generate_distillation_data.py10.1 KB
- 脚本scripts/monitor_training.py5.7 KB
- 脚本scripts/score_dataset.py7.7 KB
- 脚本scripts/submit_training.py10.8 KB
- 脚本scripts/validate/__init__.py305 B
- 脚本scripts/validate/data_stats.py6.2 KB
- 脚本scripts/validate/validate_dpo.py3.7 KB
- 脚本scripts/validate/validate_rft.py8.6 KB
- 脚本scripts/validate/validate_sft.py4.5 KB
- 参考文档workflows/dataset-creation.md3.1 KB
- 参考文档workflows/diagnose-poor-results.md2.3 KB
- 参考文档workflows/full-pipeline.md3.1 KB
- 参考文档workflows/iterative-training.md3.1 KB
- 参考文档workflows/quickstart.md4.5 KB
03 SKILL.md 正文
Fine-Tuning on Azure AI Foundry
Fine-tune models using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset prep, training, deployment, and evaluation.
When to Use
Use this sub-skill when the user asks about:
- Fine-tuning a model (SFT, DPO, or RFT)
- Preparing, validating, or formatting training data
- Submitting, monitoring, or diagnosing training jobs
- Calibrating graders or pass thresholds for RFT
- Deploying or evaluating a fine-tuned model
- Choosing between training types (SFT vs DPO vs RFT)
- Distillation, synthetic data generation, or dataset quality scoring
- Large file uploads for training data
- Cleaning up fine-tuning resources (files, deployments)
Do NOT use for: General model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).
Workflows
| Stage | Guide |
|---|---|
| Quick start | workflows/quickstart.md |
| Full pipeline | workflows/full-pipeline.md |
| Create data | workflows/dataset-creation.md |
| Iterate | workflows/iterative-training.md |
| Diagnose | workflows/diagnose-poor-results.md |
References
| Topic | File |
|---|---|
| SFT vs DPO vs RFT | references/training-types.md |
| Hyperparameters | references/hyperparameters.md |
| Data formats | references/dataset-formats.md |
| Grader design (RFT) | references/grader-design.md |
| Reward hacking | references/reward-hacking.md |
| Agentic RFT (tools) | references/agentic-rft.md |
| Deployment | references/deployment.md |
| Training curves | references/training-curves.md |
| Evaluation | references/evaluation.md |
| Vision fine-tuning | references/vision-fine-tuning.md |
| Large file uploads | references/large-file-uploads.md |
| Platform gotchas | references/platform-gotchas.md |
Scripts
| Script | Purpose |
|---|---|
scripts/submit_training.py |
Submit SFT/DPO/RFT jobs |
scripts/monitor_training.py |
Poll job until completion |
scripts/calibrate_grader.py |
Find optimal RFT pass_threshold |
scripts/check_training.py |
Analyze curves, list checkpoints |
scripts/deploy_model.py |
Deploy via ARM REST API |
scripts/evaluate_model.py |
LLM judge evaluation |
scripts/convert_dataset.py |
Convert between SFT/DPO/RFT formats |
scripts/generate_distillation_data.py |
Generate synthetic training data |
scripts/score_dataset.py |
Quality scoring on training data |
scripts/cleanup.py |
Delete old files and deployments |
scripts/validate/ |
Data validators (SFT, DPO, RFT) + stats |
Rules
- Always baseline first — evaluate the base model before fine-tuning
- Validate data before submitting — run
scripts/validate/validate_sft.py - Calibrate RFT graders — target 25-50% failure rate on the base model
- Evaluate checkpoints — don't blindly deploy the final one
- Measure token cost alongside accuracy when comparing models
Quick Reference
| Task | Command |
|---|---|
| Validate SFT data | python scripts/validate/validate_sft.py data.jsonl |
| Submit SFT job | python scripts/submit_training.py --model gpt-4.1-mini --training-file train.jsonl --validation-file val.jsonl --type sft |
| Monitor job | python scripts/monitor_training.py --job-id ftjob-xxx |
| Analyze curves | python scripts/check_training.py --job-id ftjob-xxx |
| Deploy model | python scripts/deploy_model.py --model-id ft:gpt-4.1-mini:... --name my-eval |
| Evaluate model | python scripts/evaluate_model.py --deployment-name my-eval --test-file test.jsonl |
Error Handling
| Error | Cause | Fix |
|---|---|---|
| "API version not supported" | Older openai SDK on /v1/ endpoint |
Upgrade to openai>=1.0 |
| "does not support fine-tuning with Standard TrainingType" | OSS model needs globalStandard |
Use --use-rest flag or script auto-falls back |
| Job stuck in post-training eval | Under-provisioned tool endpoint (RFT) | Scale to S2+, enable Always On |
| "DeploymentNotReady" after ARM succeeds | ARM/data-plane race condition | Delete and recreate deployment, wait 5 min |
| Content safety block at deployment | PII-dense training data | Remove problematic document types |
04 同仓库更多 Skills
rag-architect-agent
Agente Arquiteto de RAG, Embeddings & Busca Vetorial. Especialista em design e implementação de pipelines de Retrieval-Augmented Generation de alta fidelidade: chunking semântico, modelagem pgvector no PostgreSQL (Neon/Supabase), busca híbrida (cosseno + BM25), reranking, validação via TypeSafe AI e filosofia Ponytail (anti-overengineering).
rag-architecture-patterns
Padrões arquiteturais e implementação de pipelines RAG (Retrieval-Augmented Generation) de alta fidelidade para qualquer projeto ou sistema. Cobre chunking semântico, embeddings, busca híbrida (pgvector + BM25), reranking, validação via TypeSafe AI e filosofia Ponytail (anti-overengineering).
skill-repair
Use this to fix and re-install agent skills that have failed installation. This skill provides the necessary context and permissions to surgically update the `manifest.json` after a fix has been applied.
security-scan
Scan your Claude Code configuration (.claude/ directory) for security vulnerabilities, misconfigurations, and injection risks using AgentShield. Checks CLAUDE.md, settings.json, MCP servers, hooks, and agent definitions.
security-review
Use this skill when adding authentication, handling user input, working with secrets, creating API endpoints, or implementing payment/sensitive features. Provides comprehensive security checklist and patterns.
security-bounty-hunter
Hunt for exploitable, bounty-worthy security issues in repositories. Focuses on remotely reachable vulnerabilities that qualify for real reports instead of noisy local-only findings.