Environment Variables — Conventions

Apply this skill when adding, renaming, or reviewing any sglang-owned environment variable (SGLANG_*, or a legacy SGL_* alias being phased out), or when touching python/sglang/srt/environ.py.

Rule 1 — Define in the Envs class in python/sglang/srt/environ.py

All sglang-owned env vars live as EnvField descriptors on the Envs class. Never add a new os.getenv("SGLANG_..."), get_bool_env_var("SGLANG_..."), or get_int_env_var("SGLANG_...") call site — the helpers in python/sglang/srt/utils/common.py carry an explicit FIXME: move your environment variable to sglang.srt.environ and exist only for pre-existing call sites.

Group the new entry under an existing section comment (e.g. # Logging Options, # Scheduler: recv interval, # Flashinfer). Add a new section comment only when none fits — never drop a new entry at the bottom of an unrelated block.

Decision table: register in Envs or use os.getenv?

Variable	Owner	Goes through Envs?
SGLANG_*	sglang	Always. The canonical prefix for all new entries.
MOONCAKE_*, ASCEND_*, DEEP_NORMAL_*, IS_H200, USE_TRITON_W8A8_FP8_KERNEL, HF_HUB_DISABLE_XET, DISABLE_OPENAPI_DOC	Upstream/vendor alias that sglang wants to centralize	Yes — register in Envs so .get() / .override() work uniformly. Keep the upstream prefix.
CUDA_*, NCCL_*, TORCH_*, OMP_*, HF_HUB_* (raw upstream)	External tooling	No. Read with os.getenv — they're set by the launcher / driver, not by sglang.
RANK, LOCAL_RANK, WORLD_SIZE, MASTER_ADDR, MASTER_PORT, HOME, PATH	Distributed launcher / OS	No. os.getenv only.
Test runner internals (PYTEST_CURRENT_TEST, etc.)	Test framework	No. os.getenv only.

SGL_* is not a parallel valid prefix — it's a deprecated legacy alias. _convert_SGL_to_SGLANG rewrites SGL_* to SGLANG_* at import time with a DeprecationWarning. Never define a new SGL_* descriptor or os.getenv("SGL_...") call site; if you see one in code, it's tech debt to migrate.

The rule of thumb: if the value's lifecycle is owned by sglang code (we read it, we may want to override it in tests, we may want to rename it), put it in Envs. If the value is set by something outside sglang and we only consume it as-is, use os.getenv.

Rule 2 — Pick the typed descriptor

Type	Use for
EnvBool(default)	boolean flag
EnvInt(default)	integer
EnvFloat(default)	float
EnvStr(default)	string
EnvTuple(())	comma-separated list, parsed via s.split(",") and stripped

Default value:

• For a knob whose "unset" state must be distinguishable from any concrete value, use None as the default (e.g. EnvStr(None), EnvInt(None)). The descriptor handles set-to-None correctly via _set_to_none.
• For a feature flag, the default encodes the production behavior. ENABLE_FOO = EnvBool(False) means foo is off in prod; DISABLE_FOO = EnvBool(False) means foo is on in prod. See Rule 4 on picking the verb.

IntEnum for multi-state knobs

When a knob has more than two discrete states (e.g. an off / soft / strict ladder), define an IntEnum next to Envs and pass the enum member as the EnvInt default. Callers compare against the enum, not raw integers:

class ToolStrictLevel(IntEnum):
    OFF = 0
    FUNCTION = 1
    PARAMETER = 2

SGLANG_TOOL_STRICT_LEVEL = EnvInt(ToolStrictLevel.OFF)

# At the call site:
if envs.SGLANG_TOOL_STRICT_LEVEL.get() >= ToolStrictLevel.PARAMETER:
    ...

Don't pile SGLANG_ENABLE_FOO_STRICT / SGLANG_ENABLE_FOO_OFF boolean knobs that fight each other — one ordered integer is cleaner.

Rule 3 — Access via the EnvField API, never raw os.environ

from sglang.srt.environ import envs

if envs.SGLANG_FOO.get():
    ...

envs.SGLANG_FOO is the descriptor itself; its __bool__ and __len__ raise on purpose so that if envs.SGLANG_FOO: fails loudly instead of silently reading as truthy. The .get() is mandatory at every read site.

Full API surface

Method	Use
.get()	Read value (parsed). Returns default if unset, or None if explicitly set to None.
.set(value)	Set value. set(None) flips the internal _set_to_none flag so the next .get() returns None, not default.
.clear()	Unset entirely. Next .get() returns default.
.is_set()	True iff the key is present in os.environ (regardless of value, including the explicit-None case).
.override(value)	Context manager — set on enter, restore exactly what was there on exit. Use this in tests.

The _set_to_none distinction matters: clear() → is_set()=False, get()=default; set(None) → is_set()=True, get()=None. A few descriptors (e.g. SGLANG_TEST_MAX_RETRY = EnvInt(None), SGLANG_DISAGGREGATION_THREAD_POOL_SIZE = EnvInt(None)) rely on this to distinguish "user opted out" from "default applies".

Test overrides

with envs.SGLANG_TEST_RETRACT.override(True):
    ...

Don't mutate os.environ directly in tests — override restores the original cleanly, including the explicit-None state, even if the block raises.

For multiple overrides composed dynamically, use ExitStack:

with ExitStack() as stack:
    for name, value in test_envs.items():
        stack.enter_context(getattr(envs, name).override(value))
    run_test()

Subprocess inheritance

override mutates the real os.environ, so child processes spawned inside the with block inherit the override. This is the supported way to seed a subprocess.Popen:

with envs.SGLANG_TEST_RETRACT.override(True):
    subprocess.Popen([...]).wait()

A child started outside the with block sees the original value.

temp_set_env is for non-sglang keys only

The module-level temp_set_env(**env_vars) helper exists for overriding non-sglang env vars (e.g. CUDA_LAUNCH_BLOCKING, NCCL_DEBUG) in tests. It explicitly rejects SGLANG_* / SGL_* keys:

# Wrong — raises ValueError
with temp_set_env(SGLANG_TEST_RETRACT="true"): ...

# Right — sglang keys go through the descriptor
with envs.SGLANG_TEST_RETRACT.override(True): ...

# Right — non-sglang keys go through temp_set_env
with temp_set_env(CUDA_LAUNCH_BLOCKING="1"): ...

The allow_sglang=True escape hatch exists for the rare case where you must bypass Envs (e.g. setting an env var name that's only constructed at runtime); don't use it just to skip writing a descriptor.

Rule 4 — Naming: SGLANG_ prefix + verb category

Prefix is always SGLANG_ for new entries. SGL_* is auto-translated to SGLANG_* with a DeprecationWarning in _convert_SGL_to_SGLANG; never add a new SGL_* key.

The second token signals intent. Pick the right verb up front — renames require an alias entry (Rule 5).

Verb	Meaning	Example
ENABLE_FOO	Knob that turns feature foo on/off. Default in the EnvBool encodes prod behavior.	SGLANG_ENABLE_TORCH_COMPILE, SGLANG_ENABLE_OVERLAP_PLAN_STREAM
DISABLE_FOO	Kill-switch. DISABLE_FOO=True turns foo off.	SGLANG_DISABLE_CONSECUTIVE_PREFILL_OVERLAP
USE_FOO	Selects which implementation / backend	SGLANG_USE_AITER, SGLANG_USE_DEEPGEMM_BMM
FORCE_FOO	Overrides autodetection	SGLANG_FORCE_FP8_MARLIN, SGLANG_FORCE_STREAM_INTERVAL
LOG_FOO	Logging-only knob	SGLANG_LOG_GC, SGLANG_LOG_MS
TEST_FOO	Test-only hook	SGLANG_TEST_RETRACT, SGLANG_TEST_MAX_RETRY
DEBUG_FOO	Debug-only instrumentation	SGLANG_DEBUG_MEMORY_POOL, SGLANG_DEBUG_SYMM_MEM
OPT_FOO	Perf-optimization toggle (heavily used by DSV4 work)	SGLANG_OPT_USE_FUSED_HASH_TOPK, SGLANG_OPT_USE_CUSTOM_ALL_REDUCE_V2

Picking between ENABLE_FOO and DISABLE_FOO: both verbs are valid. The only forbidden combination is DISABLE_FOO = EnvBool(True), because it produces a true double-negative at the call site (if not envs.SGLANG_DISABLE_FOO.get(): reads as "if not disabled"). All other combinations are fine:

Pattern	Call site	Verdict
ENABLE_FOO = EnvBool(False)	if envs.SGLANG_ENABLE_FOO.get():	OK — opt-in feature
ENABLE_FOO = EnvBool(True)	if envs.SGLANG_ENABLE_FOO.get():	OK — on in prod, user opts out via False
DISABLE_FOO = EnvBool(False)	if not envs.SGLANG_DISABLE_FOO.get():	OK — single negation, reads as "if enabled"
DISABLE_FOO = EnvBool(True)	if not envs.SGLANG_DISABLE_FOO.get():	Forbidden — true double-negative

SGLANG_* is the canonical sglang prefix. Vendor-integration keys (MOONCAKE_*, ASCEND_*, DEEP_NORMAL_*, IS_H200) keep their upstream prefix and live in the same Envs class — these are integration aliases, not sglang-owned feature flags.

Rule 5 — Renames go through *WithAlias or _print_deprecated_env

For a rename where the old key must keep working with a warning:

SGLANG_NEW_NAME = EnvBoolWithAlias(True, deprecated_name="SGLANG_OLD_NAME")

Use EnvBoolWithAlias / EnvIntWithAlias. The fallback emits a DeprecationWarning and copies the old value over.

For a full removal where the env var is going away, add to _convert_SGL_to_SGLANG:

_print_deprecated_env("SGLANG_OLD_NAME", "SGLANG_NEW_NAME")  # mapped to a replacement
_print_deprecated_env("SGLANG_OLD_NAME")                     # no replacement, gone

For env-var to CLI-flag migration, add at module top-level:

_warn_deprecated_env_to_cli_flag(
    "SGLANG_FOO",
    "Please use '--foo' instead.",
)

Don't silently flip a default during a rename. If the new default disagrees with the old one, that's a behavior change — call it out in the PR body separately from the rename.

Rule 6 — Env var vs CLI flag

If the knob is…	Goes in
User-facing (documented, expected to flip per deployment)	server_args.py CLI flag
Expert toggle, A-B kill-switch, vendor integration	environ.py env var
Test / debug hook	environ.py env var with TEST_ / DEBUG_ prefix
Temporary env var rolling out to a CLI flag	env var first, then migrate via _warn_deprecated_env_to_cli_flag

Don't add a CLI flag that just forwards to an env var, and don't add an env var that duplicates an existing CLI flag. Pick one surface.

Out of scope

• External / vendor env vars consumed raw (HF_HUB_*, CUDA_*, NCCL_*, TORCH_*, OMP_*, RANK, MASTER_ADDR, etc.): see the decision table in Rule 1 — os.getenv is correct, don't pull them into Envs.
• Pre-existing get_bool_env_var(...) / get_int_env_var(...) call sites: leave them as is; new code shouldn't add more, but mass-migration is out of scope for a feature PR.
• Upstream-aliased keys already in Envs (MOONCAKE_*, ASCEND_*, DEEP_NORMAL_*, IS_H200, USE_TRITON_W8A8_FP8_KERNEL, HF_HUB_DISABLE_XET, DISABLE_OPENAPI_DOC — see Rule 1 decision table): the SGLANG_ prefix rules in Rule 4 don't apply — the upstream prefix is the canonical name.