Skip to content
cccskills
FreePublished on GitHub

openai-evals-tuning

OpenAI API (developers.openai.com) の Evals / Fine-tuning リファレンス。

Evals API, Datasets, graders(string check / text similarity / model / python / multi), agent evals, trace grading, external models, Promptfoo(移行先), red teaming, SFT, DPO, RFT, vision fine-tuning, distillation, prompt optimizer, fine-tuning best practices。

Evals は 2026-11-30 停止予定、セルフサービス fine-tuning は段階的廃止中。

The English summary is being prepared. Showing the original description.

Works withOpenAI
See how to install

Files included (32)

  • SKILL.md5.0 KB
  • references/evals/agent-evals.md1.7 KB
  • references/evals/evaluation-best-practices.md2.8 KB
  • references/evals/external-models.md1.9 KB
  • references/evals/getting-started-datasets.md2.8 KB
  • references/evals/graders.md4.8 KB
  • references/evals/promptfoo.md2.6 KB
  • references/evals/README.md1.6 KB
  • references/evals/red-teaming.md1.7 KB
  • references/evals/trace-grading.md1.5 KB
  • references/evals/working-with-evals.md4.4 KB
  • references/fine-tuning/direct-preference-optimization.md2.3 KB
  • references/fine-tuning/fine-tuning-best-practices.md2.6 KB
  • references/fine-tuning/model-optimization.md2.8 KB
  • references/fine-tuning/prompt-optimizer.md1.8 KB
  • references/fine-tuning/README.md1.3 KB
  • references/fine-tuning/reinforcement-fine-tuning-use-cases.md2.7 KB
  • references/fine-tuning/reinforcement-fine-tuning.md4.3 KB
  • references/fine-tuning/supervised-fine-tuning.md4.8 KB
  • references/fine-tuning/vision-fine-tuning.md2.1 KB
  • samples/distillation-workflow.md4.5 KB
  • samples/dpo-basics.md1.7 KB
  • samples/eval-create-and-run.md3.8 KB
  • samples/grader-definitions.md4.1 KB
  • samples/README.md1.3 KB
  • samples/rft-basics.md3.5 KB
  • samples/sft-dataset-and-job.md3.4 KB
  • scripts/evals.md5.3 KB
  • scripts/fine-tuning-jobs.md2.7 KB
  • scripts/promptfoo.md1.5 KB
  • scripts/README.md649 B
  • scripts/training-data-files.md1.5 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

ディレクトリ構成

skills/openai-evals-tuning/
  SKILL.md
  references/
    evals/
      README.md
      working-with-evals.md
      graders.md
      getting-started-datasets.md
      agent-evals.md
      trace-grading.md
      external-models.md
      evaluation-best-practices.md
      promptfoo.md
      red-teaming.md
    fine-tuning/
      README.md
      model-optimization.md
      supervised-fine-tuning.md
      vision-fine-tuning.md
      direct-preference-optimization.md
      reinforcement-fine-tuning.md
      reinforcement-fine-tuning-use-cases.md
      fine-tuning-best-practices.md
      prompt-optimizer.md
  samples/
    README.md
    eval-create-and-run.md
    grader-definitions.md
    sft-dataset-and-job.md
    dpo-basics.md
    rft-basics.md
    distillation-workflow.md
  scripts/
    README.md
    fine-tuning-jobs.md
    training-data-files.md
    evals.md
    promptfoo.md

探索手順

タスクからカテゴリを引き、カテゴリの README.md で目的のページを特定する:

  1. 下記マッピング表でタスクに対応するカテゴリを探す
  2. そのカテゴリの references/{category}/README.md を参照して目的のページを特定する
  3. 該当ページの .md を Read して詳細を確認する

タスク → カテゴリ マッピング

タスクカテゴリ参照 README
Evals API で eval を作成・実行したいevalsreferences/evals/README.md
grader(string check / text similarity / model / python / multi)を定義したいevalsreferences/evals/README.md
エージェントのトレースを採点・評価したいevalsreferences/evals/README.md
サードパーティモデル・外部エンドポイントを評価したいevalsreferences/evals/README.md
Evals 廃止に伴い Promptfoo へ移行したい / Promptfoo で eval を始めたいevalsreferences/evals/README.md
adversarial な入力で red teaming したいevalsreferences/evals/README.md
SFT / DPO / RFT で fine-tuning ジョブを作りたいfine-tuningreferences/fine-tuning/README.md
vision fine-tuning や distillation を行いたいfine-tuningreferences/fine-tuning/README.md
prompt optimizer の使い方を知りたいfine-tuningreferences/fine-tuning/README.md
典型的な使い方を知りたいsamplessamples/README.md
curl / Promptfoo CLI でジョブ・ファイル・eval を操作したいscriptsscripts/README.md

注意事項

  • セルフサービス fine-tuning(SFT / DPO / RFT / vision fine-tuning / distillation)は段階的廃止が進行中。新規組織での利用は 2026-05-07 以降不可、既存組織も 2027-01-06 以降は新規ジョブ作成が不可になる。各ページの ## Notes に個別の期限を記載している
  • Evals platform は 2026-06-03 に deprecated。2026-10-31 に既存 evals が read-only、2026-11-30 に Evals dashboard と API が shutdown 予定。Datasets / graders / trace grading の評価経路 / dataset-backed prompt optimizer も同じ日程が対象。移行先は OpenAI 公式推奨の Promptfoo(OSS CLI、OpenAI の一部)で、cookbook "Moving from OpenAI Evals to Promptfoo" が移行経路(references/evals/promptfoo.md / scripts/promptfoo.md)
  • fine-tuning 対応 base model の shutdown: gpt-4.1-nano-2025-04-14 / o4-mini-2025-04-16(RFT の唯一の base)は 2026-10-23、gpt-4o-2024-08-06(vision)/ gpt-4o-2024-11-20 / gpt-4o-mini-2024-07-18(蒸留サンプルの base)/ gpt-4.1-2025-04-14 / gpt-4.1-mini-2025-04-14 は 2027-01-14(deprecations ページ)。fine-tuned 版は base と同じ日程。reusable prompts(v1/prompts)と Agent Builder も 2026-11-30 に shutdown
  • distillation は専用ページが無く、SFT guide の 1 節と archived cookbook が出典(samples/distillation-workflow.md)
  • 出典: developers.openai.com/api/docs(2026-10-16 取得)
  • reasoning モデルの使い方や Responses API 自体のリファレンスは openai-api-core スキルが担当する。本スキルは評価(evals)とモデル最適化(fine-tuning)に限定する

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

AMD ROCm / HIP GPU コンピューティングリファレンス。 hipMalloc, hipLaunchKernelGGL, HIP Graph, hipGreenCtx, HIPRTC, hipify, rocBLAS, MIOpen, RCCL, hipThreads, libhipcxx, rocprofv3, amd-smi, rocminfo, TheRock / Core SDK 10, CDNA1-4 / RDNA1-4, MI300 / MI350 / MI350P, gfx942, Matrix Core, wave32/64。

The English summary is being prepared. Showing the original description.

Fandhe-AI/agent-reference-skills4Updated Oct 20, 2026

Android アプリ開発 (Kotlin) の推奨アーキテクチャリファレンス。 Jetpack, androidx, UI layer, Domain layer, Data layer, state holder, UDF, SSOT, オフラインファースト, モジュール化, ViewModel, viewModelScope, SavedStateHandle, Lifecycle, repeatOnLifecycle, collectAsStateWithLifecycle, LiveData, Hilt による DI, Paging 3 によるページング処理。

The English summary is being prepared. Showing the original description.

Fandhe-AI/agent-reference-skills4Updated Oct 20, 2026

Android アプリ開発 (Kotlin) のバックグラウンド・非同期処理リファレンス。 Jetpack androidx WorkManager: Worker, CoroutineWorker, WorkRequest, Constraints, WorkInfo, WorkContinuation, expedited work, テスト。 Service: フォアグラウンドサービス, バウンドサービス, AIDL, Messenger。 AlarmManager, PendingIntent, BroadcastReceiver, JobScheduler, Doze。 Kotlin コルーチン, Flow, StateFlow, SharedFlow, Channel。

The English summary is being prepared. Showing the original description.

Fandhe-AI/agent-reference-skills4Updated Oct 20, 2026

Android アプリ開発 (Kotlin) の Gradle ビルドシステムリファレンス。 Android Gradle Plugin (AGP), Kotlin DSL (build.gradle.kts), version catalog, KSP, R8/ProGuard, App Bundle, baseline profile。

The English summary is being prepared. Showing the original description.

Fandhe-AI/agent-reference-skills4Updated Oct 20, 2026

Android Jetpack Compose (Kotlin) の androidx Material3 コンポーネントリファレンス。 Button, IconButton, FloatingActionButton, Checkbox, Switch, Slider, TextField, SearchBar, DropdownMenu, Chip, DatePicker, TimePicker, Card, Scaffold, Dialog, BottomSheet, TopAppBar, NavigationBar, NavigationRail, Tab, Snackbar, ProgressIndicator, Badge, Icon, MaterialTheme, ColorScheme, Typography, Shapes。

The English summary is being prepared. Showing the original description.

Fandhe-AI/agent-reference-skills4Updated Oct 20, 2026

Android Jetpack Compose (Kotlin) のアニメーション・描画リファレンス。 androidx.compose.animation / graphics API。 AnimatedVisibility, AnimatedContent, animate*AsState, Animatable, AnimationSpec, Canvas, DrawScope, Brush, Path, Color, Image。

The English summary is being prepared. Showing the original description.

Fandhe-AI/agent-reference-skills4Updated Oct 20, 2026

See all skills by Fandhe-AI

Report a problem with this skill