Results for “attention optimization”

Integrate Flash Attention into transformer models to reduce processing time and GPU memory use for long sequences. Covers PyTorch integration, advanced attention options, and benchmarks that compare speed, memory use, and output accuracy.

• Speeding up transformer training on long sequences
• Reducing GPU memory use during attention
• Adding Flash Attention to a PyTorch model

Optimize transformer attention, the calculation that relates tokens to one another, using Flash Attention. The skill guides PyTorch integration, GPU memory reduction, performance measurement, and output comparisons for training or inference with long sequences.

• Reducing GPU memory use with long sequences
• Integrating Flash Attention into PyTorch models
• Comparing runtime and outputs before and after changes

Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (>512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.

The English summary is being prepared. Showing the original description.

Integrate Flash Attention into Transformer models to speed up attention, the calculation that relates input elements, while reducing GPU memory use. Covers PyTorch integration, performance measurement, output comparison, and a separate H100 optimization path.

• Optimize training with long input sequences
• Reduce memory use during long-context inference
• Measure speed and compare model outputs

Measure and optimize claude-flow v3 startup, search, attention processing, and memory use. Compare benchmarks against performance targets, track changes over time, and identify regressions to guide further optimization.

• Compare performance before and after v3 optimization
• Evaluate vector search speed and memory use
• Track performance regressions over time

Measure and optimize claude-flow v3 startup, search, attention processing, and memory use. Compare benchmarks against performance targets, track changes over time, and identify regressions to guide further optimization.

• Compare performance before and after v3 optimization
• Evaluate vector search speed and memory use
• Track performance regressions over time

Otimiza atenção em transformers com Flash Attention para ganho de 2-4x em velocidade e redução de 10-20x em memória. Use ao treinar/executar transformers com sequências longas (>512 tokens), ao encontrar problemas de memória GPU com atenção, ou quando precisa de inferência mais rápida. Suporta SDPA nativo do PyTorch, biblioteca flash-attn, H100 FP8 e sliding window attention.

The English summary is being prepared. Showing the original description.

Learn how instructions, documents, tool results, and conversation history form an AI model’s context. Understand attention limits, why information in the middle can be missed, and how relevance and placement inform decisions about what to include.

• Learning the basics of context engineering
• Understanding why more input does not always help
• Explaining how instruction placement affects attention

Helps configure and troubleshoot DeepSpeed for training large models across multiple devices. Covers memory optimization, parallel training, and DeepNVMe storage transfers, with examples for tuning data movement and configuring models that divide work among specialized components.

• Configuring distributed training for large models
• Tuning tensor transfers to NVMe storage
• Setting up models with specialized expert components

Implement and troubleshoot large-model training with DeepSpeed across multiple GPUs. The skill covers memory optimization, parallel training, and numerical precision choices, alongside examples for storage transfers and models that divide work among specialized components.

• Review settings for distributed model training
• Tune transfers between NVMe storage and tensors
• Configure training with specialized model components

Design popups, banners, and slide-in messages for email signups or purchases. Get copy, display timing, audience rules, frequency limits, mobile guidance, and test ideas, with attention to easy dismissal and accessible interaction.

• Design a newsletter signup popup for a blog
• Plan a discount offer for visitors about to leave
• Review mobile popup layout and dismissal

Train language models with SimPO using pairs of preferred and rejected answers. Configure preference training without a separate reference model, adapt settings for base or instruction models, and address unstable training, memory limits, or lost capabilities.

• Training a base model with preferred and rejected answers
• Further tuning an instruction model with preference data
• Adjusting training settings for math and code tasks

Extend the amount of text a language model can process at once. Compares RoPE, YaRN, ALiBi, and position interpolation, with implementation examples, additional training guidance, and memory optimization approaches for long document processing.

• Extend an existing model to handle longer inputs
• Compare methods for representing token positions
• Plan additional training with long documents

Helps speed up LLM text generation by comparing methods that predict and verify multiple tokens at once. Covers draft model selection, Medusa prediction heads, Lookahead Decoding settings, and deployment examples for inference services.

• Reducing chatbot response latency
• Comparing methods for faster code generation
• Selecting a small draft model for a larger LLM

Reduce LLM generation delays with speculative decoding, which proposes tokens ahead of verification. This skill compares small draft models, Medusa, and Lookahead Decoding, with implementation examples, tuning guidance, and deployment using vLLM.

• Reducing response delays in chatbots
• Speeding up code generation
• Choosing a small model to draft candidate tokens

Compress large language model weights to four bits with AWQ, a method that uses model activation patterns to limit quality loss. Follow workflows for quantization, loading compressed models, and serving them with vLLM using less GPU memory.

• Running large language models with limited GPU memory
• Quantizing a model with domain-specific text
• Serving an AWQ model with vLLM

Guide language model training with SimPO using pairs of preferred and rejected answers. Covers configuration, learning rate tuning, and memory troubleshooting for preference alignment, which adjusts model responses toward human preferences without a separate reference model.

• Training on preferred and rejected answer pairs
• Exploring an alternative to DPO without a reference model
• Tuning preference training for math and code tasks

Achieve aggressive v3 performance targets: 2.49x-7.47x Flash Attention speedup, 150x-12,500x search improvements, 50-75% memory reduction. Comprehensive benchmarking and optimization suite.

The English summary is being prepared. Showing the original description.

Achieve aggressive v3 performance targets: 2.49x-7.47x Flash Attention speedup, 150x-12,500x search improvements, 50-75% memory reduction. Comprehensive benchmarking and optimization suite.

The English summary is being prepared. Showing the original description.

Achieve aggressive v3 performance targets: 2.49x-7.47x Flash Attention speedup, 150x-12,500x search improvements, 50-75% memory reduction. Comprehensive benchmarking and optimization suite.

The English summary is being prepared. Showing the original description.

Achieve aggressive v3 performance targets: 2.49x-7.47x Flash Attention speedup, 150x-12,500x search improvements, 50-75% memory reduction. Comprehensive benchmarking and optimization suite.

The English summary is being prepared. Showing the original description.

Achieve aggressive v3 performance targets: 2.49x-7.47x Flash Attention speedup, 150x-12,500x search improvements, 50-75% memory reduction. Comprehensive benchmarking and optimization suite.

The English summary is being prepared. Showing the original description.

DiT 计算模块（L3）：把**已定位的 DiT 计算瓶颈**落成特性级选档与实施—— 量化档（W8A16 / W4A16 / W8A8 系列 / W4A4 / MXFP8 / FA 量化）、稀疏（rf_v2 / ada_bsa）、 缓存（DiTCache / AttentionCache / 时间步优化）、编译启用（MindieSDBackend / Pattern 融合 / ACLGraph） 的**开不开、开哪一档、怎么开、怎么复验**；依据是 `docs/zh/features/*`（特性真源）+ framework-integration/references/framework-support-matrix.md（支持状态）。 即使用户只说"这个模型怎么加速""量化/稀疏/Cache 怎么选怎么开""要不要开量化、开哪一档""这个档位开了有没有效果" 而未提 profiling，也应触发。 **入口条件**：瓶颈点已明确（用户带一句实测锚点，或编排层交付标签）时由域入口 `performance-optimization` 按标签分发到本技能；**瓶颈未明（"怎么加速 / 跑通 / 采 profile"）先走 `model-auto-optimization` 定位**，不在本技能内做占比分析。 near-miss：多卡并行形态 / 通信掩盖 / TP·offload 选型 → `dit-parallel-opt`；VAE 解码段与 host 固定开销 → 各自模块（VAE / host）；单算子实现级实测选型（mindie_bench）→ `benchmark-dev`； 需要新增 pattern / 算子才能落地本档 → `pattern-dev` / `operator-dev`；框架侧开关与使能验证 → `framework-integration`；量化器位级契约与精度对齐（编码公式 / 舍入 / scale 粒度）→ `quantization-dev`；精度验收判据 → `accuracy-gate`；数字入库口径 → `perf-gate`。

The English summary is being prepared. Showing the original description.

Expert guidance for distributed training with DeepSpeed - ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, sparse attention

The English summary is being prepared. Showing the original description.