본문으로 건너뛰기
cccskills
무료GitHub에 공개

test-release-canary

Run OpenShell’s Release Canary checks to verify that published development packages and Helm charts install and support basic sandbox operations. Use workflow logs and local Kubernetes reproduction steps to investigate failures before a public release.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

원문원문 설명 보기

Manually dispatch and iterate on the Release Canary workflow that smoke-tests published OpenShell artifacts (install.sh on macOS/Ubuntu/Fedora, Helm chart on kind) after each Release Dev publish. Use when changing `.github/workflows/release-canary.yml`, validating a release before tagging, debugging a canary failure, or reproducing a canary job locally. Trigger keywords - release canary, release-canary, canary failed, canary dispatch, test release canary, post-release smoke, install.sh canary, helm chart canary, kind canary, dispatch canary.

설치 방법 보기

이럴 때 유용합니다

  • Checking published development artifacts before release
  • Testing changes to the Release Canary workflow
  • Investigating installation or sandbox test failures
  • Reproducing Kubernetes checks locally with kind

포함된 파일(1)

  • SKILL.md10.6 KB

SKILL.md(원문)

설치하기 전에 에이전트에게 전달되는 지시 내용을 확인할 수 있습니다.

Test Release Canary

The Release Canary (.github/workflows/release-canary.yml) smoke-tests the artifacts a Release Dev run just published. It is the last automated checkpoint before tagging a public release: if the canary is red, the published dev artifacts do not install on a stock environment.

What the canary verifies

JobRunnerVerifies
macosmacos-latest-xlargeInstalls the dev Homebrew artifacts, reaches the VM gateway, and creates, executes in, and deletes a sandbox.
ubuntu-debubuntu-latestInstalls the dev Debian package, reaches the Docker gateway, and creates, executes in, and deletes a sandbox.
fedorafedora:latest containerInstalls the dev RPM packages, reaches the Podman gateway, and creates, executes in, and deletes a sandbox.
ubuntu-snap-system-dockerubuntu-latestUses install.sh to install the snap from latest/edge, reuses system Docker, verifies the packaged prover version and a local policy boundary check, reaches the Docker gateway, creates, executes in, and deletes a sandbox, and verifies that the Docker snap is not installed.
ubuntu-snap-docker-preflightubuntu-latestVerifies that install.sh rejects the OpenShell snap path when Docker is absent or supplied by the Docker snap, without installing OpenShell.
kubernetesubuntu-latest + kindInstalls the dev Helm chart, reaches the in-cluster gateway, and creates, executes in, and deletes a sandbox using the published runtime images.

All canary jobs disable anonymous OpenShell telemetry. Host package jobs inject OPENSHELL_TELEMETRY_ENABLED=false through the service environment, and the Kubernetes job installs with server.telemetryEnabled=false, so smoke traffic does not contribute to product usage metrics.

The workflow sets OPENSHELL_VERSION=dev for every install.sh job. Positive jobs consume the rolling dev release produced by the triggering workflow. The system-Docker snap lane tracks latest/edge; the missing-Docker lane exits before installing a snap. The Debian and Kubernetes CLI lanes remove snapd so they continue to exercise the dev Debian package. Kubernetes pins the matching 0.0.0-dev chart and :dev images.

RPM package installation also has tmachine conformance coverage in Branch E2E Checks (with test:e2e), Release Dev, Release, and Pre-release. Those lanes use the rpm installer on fedora-podman-rootful and fedora-podman-rootless with candidate CLI and gateway RPMs and matching runtime images. Branch RPM package builds run on every approved branch run, independently of optional E2E labels.

The host-package jobs exercise fresh installs, not upgrades from a persisted schema-v1 gateway config. Validate Homebrew and RPM exact-default migration with the release-tooling and package lifecycle tests before relying on the canary.

The canary does not install or import @nvidia/openshell-sdk. TypeScript SDK validation lives in the TypeScript SDK branch check, including a publish dry-run. The tagged release workflow publishes the package to GitHub Packages; verify that job directly when diagnosing SDK publication failures.

Trigger paths

The workflow has two triggers:

on:
  workflow_dispatch:
  workflow_run:
    workflows: ["Release Dev"]
    types: [completed]
  • Automatic. Every successful Release Dev run (on main or a manual dispatch of Release Dev) fires the canary. Each job gates on github.event.workflow_run.conclusion == 'success' so a failed Release Dev does not run the canary.
  • Manual. workflow_dispatch lets you run the canary on demand against any branch's workflow definition.

When dispatched manually, github.event.workflow_run.head_sha is empty and the workflow falls back to github.sha (the branch tip) for the install.sh URL.

Manual dispatch

Run the canary as-is on the current branch:

gh workflow run release-canary.yml --ref "$(git branch --show-current)"

Watch the run that starts:

sleep 5  # let GitHub register the dispatch
gh run list --workflow release-canary.yml --limit 1
gh run watch "$(gh run list --workflow release-canary.yml --limit 1 --json databaseId --jq '.[0].databaseId')"

View only failed jobs after completion:

gh run view <run-id> --log-failed

Iterating on the canary itself

When you change release-canary.yml on a branch, a manual dispatch on that branch tests your branch's workflow logic against main's published dev artifacts (0.0.0-dev chart, :dev images, and the dev GitHub release). This is what you want for iterating on the canary — you're validating that the canary still works against known-good artifacts.

Note install.sh is pulled from raw.githubusercontent.com/NVIDIA/OpenShell/${head_sha}/install.sh, so changes to install.sh on your branch are exercised even though packages come from the latest public dev release and the snap comes from latest/edge.

Testing artifacts from a specific SHA

Release Dev publishes two chart versions for every dev build (see .github/actions/release-helm-oci/action.yml:89-102):

  • oci://ghcr.io/nvidia/openshell/helm-chart:0.0.0-dev — floating, overwritten on every main push.
  • oci://ghcr.io/nvidia/openshell/helm-chart:0.0.0-dev.<sha> — immutable, appVersion set to the same SHA so it pulls the matching gateway, sandbox, and supervisor images.

To smoke-test the chart for a specific dev build, dispatch Release Dev on the branch first, then run the kind canary steps locally pointed at the SHA-pinned chart (see "Local kind reproduction" below). The release-canary workflow itself does not currently expose chart_version / image_tag inputs.

Local kind reproduction

The kubernetes job can be reproduced on any machine with Docker and mise install-provided kubectl + helm:

kind create cluster --name release-canary-local

bash e2e/support/install-agent-sandbox.sh

helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart \
  --version 0.0.0-dev \
  --namespace openshell --create-namespace \
  --set server.disableTls=true \
  --set server.telemetryEnabled=false \
  --wait --timeout 5m

kubectl wait --namespace openshell \
  --for=condition=Ready pod \
  --selector="app.kubernetes.io/name=openshell,app.kubernetes.io/instance=openshell" \
  --timeout=300s

kubectl port-forward --namespace openshell svc/openshell 8080:8080 &
openshell gateway add http://127.0.0.1:8080 --local --name kind
openshell status

Keep pkiInitJob.enabled=true (the chart default), even when server.disableTls=true. The hook also generates the sandbox JWT signing secret that the gateway pod always mounts.

Swap 0.0.0-dev for 0.0.0-dev.<sha> to pin to a specific dev build. Tear down with kind delete cluster --name release-canary-local.

Loopback registration auto-derives the gateway name to openshell if --name is omitted, which collides with the install.sh-installed local gateway — always pass --name kind (or another distinct name) when registering in addition to a local install.

Diagnosing failures

SymptomLikely causeWhere to look
macos/ubuntu-deb/fedora job fails on install.shDev release missing an asset, checksum mismatch, or install.sh regression on this branch.Job log around the curl … install.sh | sh step.
Sandbox create or exec failsPublished sandbox and supervisor artifacts are missing, incompatible, or cannot establish the protected runtime channel.Gateway logs plus Docker, Podman, VM, snap, or Kubernetes runtime diagnostics for the job.
macos/ubuntu-deb/fedora job fails on openshell statusLocal gateway service did not start (systemd/brew/podman). Often a driver issue.Service logs in the job log; OPENSHELL_COMPUTE_DRIVER env in the "Ensure …" step.
ubuntu-snap-system-docker fails during install.shSystem Docker was unavailable to the runner user, the edge revision or automatic interfaces were unavailable, or the user gateway did not become reachable.Failure diagnostics dump system Docker, snap service/connection/change state, user and legacy gateway journals, snap logs, and port 17670 listeners.
ubuntu-snap-system-docker fails during the prover checksThe prover artifact is missing or packaged for the wrong architecture, openshell.prover is not exposed or confined to read the test policies, or its solver linkage is not runnable.The Install and check status and Check a policy boundary with the snap prover steps, plus snap info openshell and snap connections openshell.
ubuntu-snap-docker-preflight unexpectedly succeedsThe installer no longer fails before installing the OpenShell snap when Docker is absent or supplied by the Docker snap.Inspect install.log, docker-snap.log, snap list, and snapd changes.
kubernetes job fails on helm install --waitChart did not deploy in 5 min — usually image pull failure or readiness probe failing."Diagnostics on failure" step dumps helm status, manifest, pod describe, pod logs.
kubernetes job fails on kubectl waitGateway pod stuck CrashLoopBackOff or ImagePullBackOff.Diagnostics dump; check :dev image existence at ghcr.io/nvidia/openshell/gateway.
kubernetes job fails on openshell gateway add or statusPort-forward not reachable, or CLI/gateway proto mismatch.port-forward.log and openshell gateway list in the diagnostics dump.

The kubernetes job's diagnostics step (only runs if: failure()) emits, in order: helm status, rendered manifest, kubectl get all, pod descriptions, pod logs (200 lines per container), port-forward log, gateway list, CLI version. Read it top-to-bottom — most failures fall out by the manifest or pod logs.

Related

  • helm-dev-environment skill — local k3d-based dev environment (more featureful than the canary's kind cluster, but uses Skaffold-built local images, not published artifacts).
  • watch-github-actions skill — generic gh run workflow monitoring.
  • debug-openshell-cluster skill — runtime gateway/sandbox diagnostics that pair with the kind job's diagnostics dump.

리뷰

아직 리뷰가 없습니다. 사용해 본 소감을 남겨 주세요.

댓글

로그인 없이 쓸 수 있습니다. 사용 후기, 질문, 작동하지 않은 환경 등을 편하게 남겨 주세요.

아직 댓글이 없습니다. 첫 댓글을 남겨 보세요.

보내면 바로 공개됩니다. 이메일 주소나 전화번호 같은 개인 정보는 쓰지 마세요. URL은 2개까지입니다.

0 / 1,000자

이 양식은 reCAPTCHA로 보호되며 Google 개인정보처리방침 및 서비스 약관이 적용됩니다.

같은 저장소의 스킬

Process one recommendation selected by ID from a local issue or CI triage report. Apply, reject, or defer it, record the outcome, and obtain separate confirmation before each change made on GitHub.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Apply one recommendation selected by its item ID
  • Reject a recommendation and record the reason
  • Defer a decision for the next queue review
NVIDIA/OpenShell1.7만2026년 10월 19일 업데이트

Turn a GitHub issue into a scoped implementation plan and, when authorized, code changes, tests, documentation, and a pull request that closes it. Check existing ownership and feedback to keep work aligned with acceptance criteria.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Creating an implementation plan from a GitHub issue
  • Implementing an authorized issue with focused tests
  • Updating documentation alongside behavior changes
NVIDIA/OpenShell1.7만2026년 10월 19일 업데이트

Maintain and validate OpenShell builds for Windows on x64 and ARM64 using MSVC, Microsoft's C++ toolchain. Check compilation, build executables, run tests on native hosts, and report artifacts and unsupported driver behavior.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Investigating Windows compilation failures
  • Building x64 and ARM64 release executables
  • Checking errors for unsupported drivers
NVIDIA/OpenShell1.7만2026년 10월 19일 업데이트

Create GitHub issues for OpenShell bugs, feature requests, and tasks using the gh command-line tool. Organize firsthand use cases, impact, and completion criteria into the project's templates, then return the created issue link.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Report an OpenShell bug you directly encountered
  • Request a feature for a specific OpenShell use case
  • Register an internal task with completion criteria
NVIDIA/OpenShell1.7만2026년 10월 19일 업데이트

Create GitHub pull requests with the gh command-line tool. Check linked issues, contributor permissions, branch conventions, and relevant tests, then prepare a title and description that follow the project's contribution rules.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Submit code changes for review
  • Open a draft pull request
  • Link a pull request to its issue
NVIDIA/OpenShell1.7만2026년 10월 19일 업데이트

Turn a design request into an OpenShell RFC, a proposal for review and discussion. The skill follows the project template, assigns an available number, and organizes motivation, alternatives, and open questions into a draft.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Drafting an architecture proposal
  • Organizing an API change proposal
  • Documenting a process change
NVIDIA/OpenShell1.7만2026년 10월 19일 업데이트

NVIDIA의 스킬 모두 보기

이 스킬의 문제 신고