finetuning

Fine-Tuning on Microsoft Foundry

Fine-tune models using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset prep, training, deployment, and evaluation.

When to Use

Use this sub-skill when the user asks about:

• Fine-tuning a model (SFT, DPO, or RFT)
• Preparing, validating, or formatting training data
• Submitting, monitoring, or diagnosing training jobs
• Calibrating graders or pass thresholds for RFT
• Deploying or evaluating a fine-tuned model
• Choosing between training types (SFT vs DPO vs RFT)
• Distillation, synthetic data generation, or dataset quality scoring
• Large file uploads for training data
• Cleaning up fine-tuning resources (files, deployments)

Do NOT use for: General model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).

Workflows

Stage	Guide
Quick start	workflows/quickstart.md
Full pipeline	workflows/full-pipeline.md
Create data	workflows/dataset-creation.md
Iterate	workflows/iterative-training.md
Diagnose	workflows/diagnose-poor-results.md

References

Topic	File
SFT vs DPO vs RFT	references/training-types.md
Hyperparameters	references/hyperparameters.md
Data formats	references/dataset-formats.md
Grader design (RFT)	references/grader-design.md
Reward hacking	references/reward-hacking.md
Agentic RFT (tools)	references/agentic-rft.md
Deployment	references/deployment.md
Training curves	references/training-curves.md
Evaluation	references/evaluation.md
Vision fine-tuning	references/vision-fine-tuning.md
Large file uploads	references/large-file-uploads.md
Platform gotchas	references/platform-gotchas.md

Scripts

Script	Purpose
scripts/submit_training.py	Submit SFT/DPO/RFT jobs
scripts/monitor_training.py	Poll job until completion
scripts/calibrate_grader.py	Find optimal RFT pass_threshold
scripts/check_training.py	Analyze curves, list checkpoints
scripts/deploy_model.py	Deploy via ARM REST API
scripts/evaluate_model.py	LLM judge evaluation
scripts/convert_dataset.py	Convert between SFT/DPO/RFT formats
scripts/generate_distillation_data.py	Generate synthetic training data
scripts/score_dataset.py	Quality scoring on training data
scripts/cleanup.py	Delete old files and deployments
scripts/validate/	Data validators (SFT, DPO, RFT) + stats

Rules

1. Always baseline first — evaluate the base model before fine-tuning
2. Validate data before submitting — run scripts/validate/validate_sft.py
3. Calibrate RFT graders — target 25-50% failure rate on the base model
4. Evaluate checkpoints — don't blindly deploy the final one
5. Measure token cost alongside accuracy when comparing models

Quick Reference

Task	Command
Validate SFT data	python scripts/validate/validate_sft.py data.jsonl
Submit SFT job	python scripts/submit_training.py --model gpt-4.1-mini --training-file train.jsonl --validation-file val.jsonl --type sft
Monitor job	python scripts/monitor_training.py --job-id ftjob-xxx
Analyze curves	python scripts/check_training.py --job-id ftjob-xxx
Deploy model	python scripts/deploy_model.py --model-id ft:gpt-4.1-mini:... --name my-eval
Evaluate model	python scripts/evaluate_model.py --deployment-name my-eval --test-file test.jsonl

Error Handling

Error	Cause	Fix
"API version not supported"	Older openai SDK on /v1/ endpoint	Upgrade to openai>=1.0
"does not support fine-tuning with Standard TrainingType"	OSS model needs globalStandard	Use --use-rest flag or script auto-falls back
Job stuck in post-training eval	Under-provisioned tool endpoint (RFT)	Scale to S2+, enable Always On
"DeploymentNotReady" after ARM succeeds	ARM/data-plane race condition	Delete and recreate deployment, wait 5 min
Content safety block at deployment	PII-dense training data	Remove problematic document types