'sglang' 검색 결과

Investigate recurring SGLang automated test failures by comparing logs, commit history, and GPU environments. Narrow down code changes or machine-specific problems, optionally reproduce failures remotely, and summarize the evidence in a structured report.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Finding the change behind a recurring test failure
• Checking whether failures depend on a GPU or runner
• Reproducing a test failure on a remote GPU server

Investigate CUDA crashes in SGLang using logs and inputs saved before execution. Inspect tensor shapes, data types, and numerical anomalies, with guidance for separate process logs and additional CUDA debugging tools.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigating illegal memory access and device-side assertions
• Tracing NaN or Inf values through kernel calls
• Saving inputs before a CUDA crash

Set up reinforcement learning, training guided by rewards, for language models with slime. The skill covers Megatron-LM training, SGLang response generation, data preparation, asynchronous execution, and custom workflows for agents that use tools across multiple turns.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Prepare GRPO training for GLM or Qwen3 models
• Overlap response generation and model training
• Train agents with multi-turn tool use

Monitor selected GitHub Actions checks on an SGLang pull request until they pass on its latest commit. Diagnose failures, push small validated fixes, and retry only failed jobs within defined limits.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Monitor selected PR checks until they pass
• Separate code failures from infrastructure problems
• Apply small fixes and track the resulting checks

Review SGLang Cookbook pull requests for configuration errors, documentation mismatches, benchmark issues, and broken links. Get findings for each file and an overall recommendation on whether to approve, request changes, or block the contribution.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Reviewing a new model's Cookbook contribution
• Checking documentation against generated launch commands
• Checking benchmark definitions and measurement consistency

Guide reinforcement learning, training models through rewards, with slime. Prepare data, configure Megatron-LM training and SGLang response generation, and set up asynchronous runs or custom multi-turn workflows for supported language models.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Preparing reward-based training for GLM or Qwen models
• Overlapping response generation and training
• Training agents that use tools across multiple turns

Set up LLM serving with structured responses, including JSON and outputs restricted to specified patterns. Covers reusing shared prompt calculations, tool calling, and multi-turn conversations through SGLang, with examples for server setup and application integration.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Extract information into JSON with a defined schema
• Build agents that call tools
• Reuse shared prompts across chat requests

Build LLM serving workflows that produce JSON or other required formats. The skill covers reusing computation for shared prompts, tool calls, and ongoing conversations, with examples for server setup and structured output.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Extracting article details into JSON
• Building agents that call tools
• Reusing shared prompts across conversations

Analyze LLM serving traces to identify GPU time usage, opportunities to run operations concurrently, and known patterns for combining operations. Produce three consistent tables from saved traces or live captures on supported servers.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Identify GPU operations that dominate inference time
• Compare input processing and token generation
• Trace GPU operations back to Python code

Add a CUDA/C++ GPU operation to sgl-kernel using AOT, meaning compilation before use. Follow the implementation through PyTorch registration, Python API exposure, SGLang integration, correctness tests, and benchmarks that account for cache reuse.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Adding a CUDA/C++ operation to sgl-kernel
• Registering an operation with PyTorch and exposing a Python API
• Testing numerical results and invalid inputs

Guides large language model reinforcement learning with miles, covering environment setup, low-precision training, consistency between training and inference, and speculative decoding, where a smaller model proposes tokens for a larger model to verify.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Configure reinforcement learning for large MoE models
• Investigate instability in FP8 training
• Set up speculative decoding for faster rollout generation

Diagnose failed model requests from OpenShell sandboxes to hosted APIs or local servers such as Ollama and vLLM. Check provider attachment, access rules, credentials, network layout, and client settings to identify the narrowest configuration change needed.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Diagnose failed requests to hosted model APIs
• Troubleshoot connections to Ollama or LM Studio
• Investigate credential and endpoint mismatch errors

Attribute GPU time in one sglang-omni stage to specific lines of `sglang_omni/` python source. Drives the `llm-torch-profiler-analysis` backend with a mapping/formal trace pair plus steady-state gates, and carries the rules for reading its three tables on an omni stage. Use when a stage is slower than expected and the question is which python code is the bottleneck, not which kernel is hot.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Calculate how to divide serving memory between recurrent state and KV cache, two memory pools used by hybrid models. Use startup logs and workload settings to recommend a ratio, estimate concurrency, and identify the limiting pool.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Choosing a memory ratio for a hybrid model
• Investigating limits on concurrent requests
• Recalculating memory allocation for context length

Guides large language model reinforcement learning with miles, including MoE models that route work to specialized components. Covers low-precision training, matching training and inference behavior, and accelerating training response generation with draft models.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up reinforcement learning for large MoE models
• Configuring FP8 or INT4 training
• Checking consistency between training and inference

Use when you need to CAPTURE / PRODUCE XProf (jax.profiler) profiling artifacts from sgl-jax (SGLang on JAX/TPU) for later performance analysis: driving a live server (/start_profile, stage-separated prefill/decode), offline bench_one_batch --profile, or kernel microbench. This skill produces the trace files; it does not analyze them.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Fornece orientação para pós-treinamento de LLM com RL usando slime, um framework Megatron+SGLang. Use ao treinar modelos GLM, implementar fluxos de trabalho personalizados de geração de dados, ou precisar de integração estreita com Megatron-LM para escalabilidade em RL.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Geração estruturada rápida e serving para LLMs com RadixAttention para cache de prefixos. Use para saídas JSON/regex, decodificação restrita, workflows com agentes que fazem chamadas de ferramentas, ou quando você precisa de 5× mais rápido que vLLM com compartilhamento de prefixos. Alimenta 300.000+ GPUs na xAI, AMD, NVIDIA e LinkedIn.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Provides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Review existing tests and CI, the automated checks run during development, for redundant coverage, misplaced tests, and opportunities to reduce runtime or cost. Receive file-specific recommendations based on documented improvement patterns, with uncertain findings separated.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigating a slow test suite
• Finding redundant or misplaced tests
• Reviewing CI runtime and cost

Verify file splits, function moves, module extractions, and renames by reproducing changes from a base commit and comparing files byte for byte. Generate repeatable proof scripts and reports that expose changes beyond relocation.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Splitting a refactor into preparation, movement, and follow-up commits
• Generating repeatable proof scripts for code moves
• Verifying every provable commit in a branch

Guides LLM training with verl using rewards to improve model responses. Covers data preparation, reward functions, GRPO and PPO configuration, distributed GPU training, and troubleshooting memory shortages or unstable training.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Train math reasoning models with GRPO and answer-based rewards
• Configure PPO training with a separate value-estimation model
• Run large-model training across multiple GPU nodes

Choose a serving container for models deployed on SageMaker. Match the model type to a compatible image, retrieve its address from the AWS catalog, and check regional availability, CPU or GPU variants, and required host settings.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Choose a container for an LLM deployment
• Select images for embeddings and rerankers
• Check regional image addresses and GPU compatibility