FreePublished on GitHub

finetuning

Fine-tune models on Microsoft Foundry using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset preparation, training job submission, deployment, and evaluation. USE FOR: fine-tune, SFT, DPO, RFT, training data, grader, distillation, fine-tuned model, training job, large file upload, calibrate grader, deploy fine-tuned model, evaluate fine-tuned model. DO NOT USE FOR: general model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).

The English summary is being prepared. Showing the original description.

microsoft/skills

Works withOpenAI

See how to install

Files included (34)

SKILL.md5.4 KB
references/agentic-rft.md3.2 KB
references/dataset-formats.md3.7 KB
references/deployment.md3.4 KB
references/evaluation.md5.8 KB
references/grader-design.md3.4 KB
references/hyperparameters.md3.2 KB
references/large-file-uploads.md927 B
references/platform-gotchas.md2.0 KB
references/reward-hacking.md2.9 KB
references/training-curves.md4.1 KB
references/training-types.md3.1 KB
references/vision-fine-tuning.md4.1 KB
scripts/calibrate_grader.py9.3 KB
scripts/check_training.py7.8 KB
scripts/cleanup.py9.5 KB
scripts/common.py7.6 KB
scripts/convert_dataset.py10.7 KB
scripts/deploy_model.py9.4 KB
scripts/evaluate_model.py11.4 KB
scripts/generate_distillation_data.py10.1 KB
scripts/monitor_training.py5.7 KB
scripts/score_dataset.py7.7 KB
scripts/submit_training.py10.8 KB
scripts/validate/__init__.py305 B
scripts/validate/data_stats.py6.2 KB
scripts/validate/validate_dpo.py3.7 KB
scripts/validate/validate_rft.py8.6 KB
scripts/validate/validate_sft.py4.5 KB
workflows/dataset-creation.md3.1 KB
workflows/diagnose-poor-results.md2.3 KB
workflows/full-pipeline.md3.1 KB
workflows/iterative-training.md3.1 KB
workflows/quickstart.md4.5 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

Fine-Tuning on Microsoft Foundry

Fine-tune models using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset prep, training, deployment, and evaluation.

When to Use

Use this sub-skill when the user asks about:

Fine-tuning a model (SFT, DPO, or RFT)
Preparing, validating, or formatting training data
Submitting, monitoring, or diagnosing training jobs
Calibrating graders or pass thresholds for RFT
Deploying or evaluating a fine-tuned model
Choosing between training types (SFT vs DPO vs RFT)
Distillation, synthetic data generation, or dataset quality scoring
Large file uploads for training data
Cleaning up fine-tuning resources (files, deployments)

Do NOT use for: General model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).

Workflows

Stage	Guide
Quick start	workflows/quickstart.md
Full pipeline	workflows/full-pipeline.md
Create data	workflows/dataset-creation.md
Iterate	workflows/iterative-training.md
Diagnose	workflows/diagnose-poor-results.md

References

Topic	File
SFT vs DPO vs RFT	references/training-types.md
Hyperparameters	references/hyperparameters.md
Data formats	references/dataset-formats.md
Grader design (RFT)	references/grader-design.md
Reward hacking	references/reward-hacking.md
Agentic RFT (tools)	references/agentic-rft.md
Deployment	references/deployment.md
Training curves	references/training-curves.md
Evaluation	references/evaluation.md
Vision fine-tuning	references/vision-fine-tuning.md
Large file uploads	references/large-file-uploads.md
Platform gotchas	references/platform-gotchas.md

Scripts

Script	Purpose
`scripts/submit_training.py`	Submit SFT/DPO/RFT jobs
`scripts/monitor_training.py`	Poll job until completion
`scripts/calibrate_grader.py`	Find optimal RFT pass_threshold
`scripts/check_training.py`	Analyze curves, list checkpoints
`scripts/deploy_model.py`	Deploy via ARM REST API
`scripts/evaluate_model.py`	LLM judge evaluation
`scripts/convert_dataset.py`	Convert between SFT/DPO/RFT formats
`scripts/generate_distillation_data.py`	Generate synthetic training data
`scripts/score_dataset.py`	Quality scoring on training data
`scripts/cleanup.py`	Delete old files and deployments
`scripts/validate/`	Data validators (SFT, DPO, RFT) + stats

Rules

Always baseline first — evaluate the base model before fine-tuning
Validate data before submitting — run scripts/validate/validate_sft.py
Calibrate RFT graders — target 25-50% failure rate on the base model
Evaluate checkpoints — don't blindly deploy the final one
Measure token cost alongside accuracy when comparing models

Quick Reference

Task	Command
Validate SFT data	`python scripts/validate/validate_sft.py data.jsonl`
Submit SFT job	`python scripts/submit_training.py --model gpt-4.1-mini --training-file train.jsonl --validation-file val.jsonl --type sft`
Monitor job	`python scripts/monitor_training.py --job-id ftjob-xxx`
Analyze curves	`python scripts/check_training.py --job-id ftjob-xxx`
Deploy model	`python scripts/deploy_model.py --model-id ft:gpt-4.1-mini:... --name my-eval`
Evaluate model	`python scripts/evaluate_model.py --deployment-name my-eval --test-file test.jsonl`

Error Handling

Error	Cause	Fix
"API version not supported"	Older `openai` SDK on `/v1/` endpoint	Upgrade to `openai>=1.0`
"does not support fine-tuning with Standard TrainingType"	OSS model needs `globalStandard`	Use `--use-rest` flag or script auto-falls back
Job stuck in post-training eval	Under-provisioned tool endpoint (RFT)	Scale to S2+, enable Always On
"DeploymentNotReady" after ARM succeeds	ARM/data-plane race condition	Delete and recreate deployment, wait 5 min
Content safety block at deployment	PII-dense training data	Remove problematic document types

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

More skills from this repository

agent-framework-azure-ai-py

Free

Build Azure AI Foundry agents using the Microsoft Agent Framework Python SDK (agent-framework-azure-ai). Use when creating persistent agents with AzureAIAgentsProvider, using hosted tools (code interpreter, file search, web search), integrating MCP servers, managing conversation threads, or implementing streaming responses. Covers function tools, structured outputs, and multi-tool agents.

The English summary is being prepared. Showing the original description.

microsoft/skills3.1KUpdated Oct 18, 2026

airunway-aks-setup

Free

Set up AI Runway on AKS — from bare cluster to running model. Covers cluster verification, controller install, GPU assessment, provider setup, and first deployment. WHEN: "setup AI Runway", "onboard AKS cluster", "install AI Runway", "airunway setup", "deploy model to AKS", "GPU inference on AKS", "KAITO setup on AKS", "run LLM on AKS", "vLLM on AKS", "set up model serving on AKS", "AI Runway controller".

The English summary is being prepared. Showing the original description.

microsoft/skills3.1KUpdated Oct 18, 2026

aks-gpu-inference

Free

Diagnose Day-2 AKS GPU and KAITO incidents using profile-aware, read-only evidence. WHEN: 'Insufficient nvidia.com/gpu', GPU pod Pending, model-load OOM, DCGM/VRAM, KAITO Workspace not ready, or GPU autoscaling. DO NOT USE FOR: setup (airunway-aks-setup), non-GPU incidents (aks-troubleshooting), standalone VM quota (azure-quotas), or generic cost (cost-analysis or cost-optimization from the optional azure-cost plugin).

The English summary is being prepared. Showing the original description.

microsoft/skills3.1KUpdated Oct 18, 2026

aks-known-issues

Free

Lookup documented AKS fixes only when the prompt includes an exact catalog signature and all of its qualifiers: VMCannotFitEphemeralOSDisk; NodePoolMcVersionIncompatible; 'NodeImageVersion is not accepted'; AKS SkuNotAvailable with size, location, and zone; ZonalAllocationFailed with insufficient zone capacity; OverconstrainedAllocationRequest with listed constraints; nested AKS vmssCSE/CSE VMExtensionError_OutboundConnFail, VMExtensionError_K8SAPIServerConnFail, or VMExtensionError_K8SAPIServerDNSLookupFail; or AllocationFailed with the full cataloged internal-error or insufficient-regional-capacity message. Never use for quota errors, code-only or bare wrappers, generic symptoms, incomplete signatures, or failures outside AKS; use aks-troubleshooting or azure-diagnostics.

The English summary is being prepared. Showing the original description.

microsoft/skills3.1KUpdated Oct 18, 2026

aks-network-capture

Free

Collects bounded packet captures from AKS nodes and Azure network configuration for wire-level evidence. WHEN: "capture packets on an AKS node", "take a pcap", "run tcpdump on AKS", "prove where packets drop". Use for explicit packet-capture intent after read-only diagnostics, not general AKS connectivity or ingress troubleshooting.

The English summary is being prepared. Showing the original description.

microsoft/skills3.1KUpdated Oct 18, 2026

aks-troubleshooting

Free

Debug live Azure Kubernetes Service (AKS) incidents with a read-only, evidence-first investigation. WHEN: pod crashes or Pending, CrashLoopBackOff, OOMKilled, ImagePullBackOff, node NotReady, DNS or ingress failure, connectivity timeout, network policy, SNAT exhaustion, node-pool scaling blocked by QuotaExceeded or InsufficientVCPUQuota, upgrade stuck, spot or zone disruption, a bare VMExtensionProvisioningError or AllocationFailed wrapper, an uncataloged capacity symptom, or 'investigate my AKS cluster'. DO NOT USE FOR: packet capture (use aks-network-capture); GPU or model-serving issues (use aks-gpu-inference); cluster creation or provisioning (use azure-kubernetes); cost (use cost-analysis from the optional azure-cost plugin); pod rightsizing (use azure-kubernetes); a fully qualified documented AKS signature with every required nested qualifier (use aks-known-issues); standalone failures on non-AKS Azure resources (use azure-diagnostics). Unqualified errors and open-ended incidents stay here only for AKS.

The English summary is being prepared. Showing the original description.

microsoft/skills3.1KUpdated Oct 18, 2026

See all skills by microsoft

Report a problem with this skill

Install

Install with a command

npx skills add https://github.com/microsoft/skills --skill finetuning

Paste it into your terminal and run it (requires Node.js). You can choose which agents to install it for, such as Claude Code or Codex.

Ask your agent

Please install the following Agent Skill.
https://github.com/microsoft/skills/tree/main/.github/plugins/azure-skills/skills/microsoft-foundry/finetuning

First review SKILL.md and the bundled files and confirm there is nothing suspicious (such as sending data externally or reading credentials). Then place it in ~/.claude/skills/finetuning/ for Claude Code, or ~/.agents/skills/finetuning/ for Codex.

Send this text to Claude Code or Codex as is. It checks the contents before installing.

View source on GitHub

Repository: microsoft/skills
Stars: 3,131
Forks: 353
Category: DevOps
License: MIT
Original language: English
Last updated: Oct 18, 2026

A skill is a set of instructions and scripts for an agent. cccskills does not guarantee that skills are safe. Review the contents before you install.