44 skills (repositories by sgl-project) · sorted by popularity
Add or verify model support in SGLang's Rust processor so chat requests produce the same prompt text and token IDs as Python. Use reference-generated test cases to find differences in tools, reasoning settings, conversation history, and errors.
Integrate or upgrade cache-dit in SGLang diffusion, the image generation system. Follow guidance for resolving code conflicts, adding cache settings, testing feature combinations, and comparing generated images using quality measurements and logs.
Choose SGLang Diffusion settings for a specific model, GPU, and memory budget. Compare generation speed, memory use, and output quality to select command-line options, including CPU offloading, multiple GPUs, caching, and reduced precision.
Add a CUDA/C++ GPU operation to sgl-kernel using AOT, meaning compilation before use. Follow the implementation through PyTorch registration, Python API exposure, SGLang integration, correctness tests, and benchmarks that account for cache reuse.
Monitor selected GitHub Actions checks on an SGLang pull request until they pass on its latest commit. Diagnose failures, push small validated fixes, and retry only failed jobs within defined limits.
Review existing tests and CI, the automated checks run during development, for redundant coverage, misplaced tests, and opportunities to reduce runtime or cost. Receive file-specific recommendations based on documented improvement patterns, with uncertain findings separated.
Understand and modify SGLang's CI pipelines, including test stage order, start conditions, early stopping after failures, and workload splitting. Use the guide to add jobs and investigate failed, skipped, or cancelled checks.
Collect and review SGLang startup logs to identify unnecessary output and trace its source. Get saved logs and specific cleanup proposals while preserving useful diagnostics, with runtime changes left for user approval.
Calculate how to divide serving memory between recurrent state and KV cache, two memory pools used by hybrid models. Use startup logs and workload settings to recommend a ratio, estimate concurrency, and identify the limiting pool.
Add a model page to the SGLang Cookbook using shared templates. Organize verified launch settings and measured benchmarks, create configuration files and documentation, and update navigation, vendor cards, and page validation.
Review SGLang Cookbook pull requests for configuration errors, documentation mismatches, benchmark issues, and broken links. Get findings for each file and an overall recommendation on whether to approve, request changes, or block the contribution.
Add, rename, or review SGLang environment variables using consistent definitions, types, names, and access methods. The skill also guides migration from old names and temporary test overrides that restore the original values afterward.
Launch an SGLang server, run a quick accuracy check, and capture an execution trace for performance inspection. Return the trace location and generated files, ready to open in Perfetto or Chrome tracing.
Organize SGLang GPU operations when adding or moving code. It guides API placement, operation registration, test organization, and migration checks to preserve existing behavior, references, and continuous integration coverage.
Build and debug sglang tests comparing token probabilities during prompt processing and step-by-step generation. Separate numerical differences from reused-state bugs, choose test thresholds, and locate the first operation that diverges.
Guide changes and reviews for SGLang’s Scheduler, TokenizerManager, and ModelRunner classes. Keep initialization in small, replaceable helpers and separate computation from coordination so large classes retain clear responsibilities and explicit dependencies.
Analyze LLM serving traces to identify GPU time usage, opportunities to run operations concurrently, and known patterns for combining operations. Produce three consistent tables from saved traces or live captures on supported servers.
Verify file splits, function moves, module extractions, and renames by reproducing changes from a base commit and comparing files byte for byte. Generate repeatable proof scripts and reports that expose changes beyond relocation.
Guide test design for SGLang's scripted runtime by deciding when a test harness API adds real value. Favor direct reads, meaningful outcome checks, and the engine's actual execution loop over unnecessary wrappers.
Investigate recurring SGLang automated test failures by comparing logs, commit history, and GPU environments. Narrow down code changes or machine-specific problems, optionally reproduce failures remotely, and summarize the evidence in a structured report.
Prepare batches of merged SGLang pull requests for a release branch. It checks conflicts and previous attempts, starts the cherry-pick workflow, monitors completion, and reports created pull requests or failure reasons.
Guide changes to SGLang runtime configuration and shared process state. Identify where resolved settings, runtime flags, resources, and per-forward state belong, and use the documented access patterns and checks when developing or testing.
Apply consistent names to SGLang speculative decoding code, which verifies draft tokens to generate text. Distinguish tokens with or without the bonus token, counts, counters, and arrays when adding identifiers, renaming code, or reviewing changes.
Write and register SGLang unit and end-to-end tests, from choosing models and file locations to reusing server fixtures. Follow project rules for meaningful assertions, CI registration, dependency mocking, and safe server cleanup after tests.