본문으로 건너뛰기
cccskills
무료GitHub에 공개

sam3sapiens2-pose

Use when working on the SAM3 plus Sapiens2 pose estimation pipeline, batch GPU processing, or VRAM and throughput issues.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vaila-multimodaltoolbox/vaila
설치 방법 보기

포함된 파일(1)

  • SKILL.md3.0 KB

SKILL.md(원문)

설치하기 전에 에이전트에게 전달되는 지시 내용을 확인할 수 있습니다.

SAM3-Guided Sapiens2 Pose Pipeline (vailá)

Use when working on SAM3 + Sapiens2 pose estimation, running batch CLI video pipelines (vaila/sam3sapiens2.py), tuning batch sizes for NVIDIA GPUs (RTX 4090 / 3090 / 5050), configuring headless GPU mode via gpumode, or diagnosing VRAM/throughput bottlenecks.

Overview

vaila/sam3sapiens2.py is a top-down markerless pose pipeline integrating:

  • SAM3 (Segment Anything Model 3): Serves as the authority for person detection, tracking, silhouette extraction, and persistent obj_ids.
  • Sapiens2 Pose (1B / 5B / 0.8b / 0.4b ViT): Runs top-down on SAM3-generated crops and masks (308 body/face/hand keypoints). DETR detector is disabled.

CLI Execution Pattern

# Standard batch run on directory:
uv run python -u vaila/sam3sapiens2.py \
    -i /path/to/videos \
    -o /path/to/output_dir \
    -t person \
    --model 1b \
    --stride 1 \
    --device 0 \
    --kpt-thr 0.3 \
    --bbox-padding 0.12 \
    --contour-margin 8 \
    --max-persons 1 \
    --pose-batch-size 16

# With Test-Time Augmentation (Flip-Test) for highest joint accuracy:
uv run python -u vaila/sam3sapiens2.py \
    -i /path/to/videos \
    -o /path/to/output_dir \
    -t person \
    --model 1b \
    --pose-batch-size 16 \
    --flip-test

GPU & Batch Sizing Benchmark (RTX 4090 24GB)

Measured on Sapiens2 1B (308 keypoints, 1024x768 crop resolution):

Batch SizeThroughput (crops/s)VRAM AllocatedVRAM Reserved (Peak)Safety Margin
42.45 crops/s7.93 GiB9.57 GiB~14.4 GiB free
82.67 crops/s10.33 GiB13.54 GiB~10.5 GiB free
122.66 crops/s12.72 GiB17.40 GiB~6.6 GiB free
162.67 crops/s15.11 GiB21.22 GiB~2.8 GiB free
202.61 crops/s17.51 GiB20.01 GiB~4.0 GiB free
242.58 crops/s18.68 GiB22.73 GiB~1.3 GiB free

Key Rules & Recommendations:

  1. Saturation: ViT 1B saturates GPU Tensor Cores at Batch 8 to 16 (~2.67 crops/s). Higher batch sizes (e.g. 24) do not increase throughput and risk CUDA OOM during long multi-video runs.
  2. Optimal Batch: Use --pose-batch-size 16 for high throughput with sufficient headroom.
  3. Single Athlete Videos: In videos with 1 athlete, batch size per frame is 1 crop (~7.19 GiB VRAM). --max-persons 1 prevents spurious multi-crop overhead.
  4. --flip-test Impact: Doubles forward passes (2x compute time), negligible VRAM change, provides higher anatomical accuracy for challenging sports movements.

Headless / CLI GPU Management (gpumode)

To free all VRAM from desktop compositors (Xorg/Wayland) on dedicated NVIDIA setups:

# Switch to CLI mode (multi-user.target, frees 100% VRAM):
sudo gpumode --cuda

# Return to GUI desktop:
sudo gpumode --desktop

리뷰

아직 리뷰가 없습니다. 사용해 본 소감을 남겨 주세요.

댓글

로그인 없이 쓸 수 있습니다. 사용 후기, 질문, 작동하지 않은 환경 등을 편하게 남겨 주세요.

아직 댓글이 없습니다. 첫 댓글을 남겨 보세요.

보내면 바로 공개됩니다. 이메일 주소나 전화번호 같은 개인 정보는 쓰지 마세요. URL은 2개까지입니다.

0 / 1,000자

이 양식은 reCAPTCHA로 보호되며 Google 개인정보처리방침 및 서비스 약관이 적용됩니다.

같은 저장소의 스킬

agent-platform-alert-configuration

무료

Configures best-practice alerting policies for AI agents using OpenTelemetry (OTel) metrics, generating output as Terraform (.tf) configuration files. Use when analyzing, writing, or deploying alerting policies to monitor agent latency, error rates, token usage, and quality metrics. Don't use for standard infrastructure monitoring unrelated to AI agents, or when the agent is not instrumented with OpenTelemetry (for Reliability, Cost, Safety, Security alerts). NOTE: Reliability, Cost, Safety, and Security alerts use generic OTel metrics and work across runtimes (such as Cloud Run, Vertex AI). Quality alerts rely on Vertex AI Online Monitors and are strictly bound to Vertex AI deployments.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vaila-multimodaltoolbox/vaila212026년 10월 20일 업데이트

agent-platform-deploy

무료

Deploy open models or custom weights from Model Garden to Agent Platform endpoints, check the status of an in-progress deployment operation, or clean up resources by undeploying models and deleting endpoints. Use when asked to actively deploy a model, list the Model Garden CATALOG of available models, check if a specific model is deployable (`gcloud ai model-garden models list-deployment-config`), query deployment cost, troubleshoot deployment errors (like quota limits), or undeploy/clean up endpoints. Also use when copying and deploying a 1P Tuned Model. Don't use for pure listing/discovery questions of the form "is X deployed?", "list my endpoints", or "which regions have models running?" — for those use `agent-platform-endpoint-management`. Don't use for public Vertex AI deployments (use the `vertex-deploy` skill) or for running model evaluations (use the `agent-platform-eval-flywheel` skill).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vaila-multimodaltoolbox/vaila212026년 10월 20일 업데이트

agent-platform-endpoint-management

무료

Manages Agent Platform serving endpoints. Use when you need to create, list, describe, update, or delete serving endpoints for model deployment on Agent Platform. Also use when troubleshooting endpoint permission, quota, or resource busy errors. Don't use for deploying models to endpoints or for running model evaluations.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vaila-multimodaltoolbox/vaila212026년 10월 20일 업데이트

agent-platform-eval-flywheel

무료

Measures and improves the quality of AI models and agents on Google Cloud using the Eval Quality Flywheel methodology. Use when evaluating an agent or model, building an eval dataset, picking or writing evaluation metrics, analyzing failures, comparing results before and after a fix, or when guidance is needed on Agent Platform eval methodology — including dataset schema, LLM-as-judge scoring, and common failure causes. For fine-tuning, use agent-platform-tuning. For general production deployment, use agent-platform-deploy.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vaila-multimodaltoolbox/vaila212026년 10월 20일 업데이트

agent-platform-inference

무료

Connects to and performs inference with Google Cloud Agent Platform GenAI models, including First-Party Gemini models and Third-Party OpenMaaS models (Llama, DeepSeek, Qwen, etc.). Use when asked to perform inference, ask a model a question, run a test prompt, execute chat completions, or generate code for calling Gemini or OpenMaaS models, authenticate with GenAI SDK, OpenAI SDK, or legacy Agent Platform SDK, configure base URLs and global/regional endpoints, or troubleshoot 429 Resource Exhausted (DSQ), 400 User Validation, or 404 Not Found errors. Don't use for deploying models to endpoints or for running model evaluations.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vaila-multimodaltoolbox/vaila212026년 10월 20일 업데이트

agent-platform-migrate-from-ai-studio

무료

Guides agents and users through migrating from Gemini API in Google AI Studio to Gemini Enterprise Agent Platform (formerly Vertex AI). Use this skill when moving applications to Google Cloud, to leverage Cloud credits, or to unify inferencing with other Cloud infrastructure (IAM, billing, telemetry).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vaila-multimodaltoolbox/vaila212026년 10월 20일 업데이트

vaila-multimodaltoolbox의 스킬 모두 보기

이 스킬의 문제 신고