Results for “teacher-student”

Choose what kind of knowledge to transfer between teacher and student models: response, feature, or relational, and decide among offline, online, self, or cross-modal distillation schemes. Best for distillation strategy selection, capacity-gap diagnosis, and transfer planning. Activate on "knowledge distillation", "teacher-student", "soft labels", "feature distillation", "online distillation", or "cross-modal transfer". NOT for generic compression checklists or unrelated training work.

The English summary is being prepared. Showing the original description.

Train smaller language models using knowledge distillation, where a larger model supplies examples or prediction targets. This skill covers training losses, generated training data, parameter choices, and output comparisons to support model compression and specialized student models.

• Training a smaller model from a larger teacher
• Fine-tuning with teacher-generated responses
• Distilling domain knowledge into a specialized model

Guide knowledge distillation, where a smaller language model learns from a larger teacher. Covers training from teacher responses or prediction scores, adjusting training settings, and comparing outputs to support model compression and specialized training.

• Compressing a large language model into a smaller model
• Training a specialized model from teacher-generated examples
• Comparing distillation methods and training settings

Interactive demo: Generate pseudo-labels from a strong teacher model, build a cheap student function, and evaluate accuracy. Showcases pseudo-labeling and teacher-student distillation.

The English summary is being prepared. Showing the original description.

Distill large models into smaller ones — teacher-student training, data generation, loss design, and quality validation. Use when you need big-model behavior at small-model cost.

The English summary is being prepared. Showing the original description.

Design deployment-focused distillation systems that balance model size, accuracy, calibration, and cascade escalation under real resource limits. Best for teacher-student compression, threshold design, and failure-aware deployment. Activate on "model compression", "teacher- student", "distillation score", "cascade model", "edge deployment", or "model calibration". NOT for generic deep-learning overviews, prompt optimization, or training work without a concrete distillation objective.

The English summary is being prepared. Showing the original description.

Comprima modelos de linguagem grandes usando destilação de conhecimento de modelos professor para aluno. Use ao implantar modelos menores com desempenho retido, transferir capacidades do GPT-4 para modelos de código aberto ou reduzir custos de inferência. Aborda escalamento de temperatura, alvos suaves, KLD reversa, destilação de logits e estratégias de treinamento MiniLLM.

The English summary is being prepared. Showing the original description.

Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.

The English summary is being prepared. Showing the original description.