bats-testing

Overview

Per bats-readthedocs:

"Bats (Bash Automated Testing System) is a TAP-compliant testing framework for Bash 3.2 or above."

Per bats-github:

"Bats is a TAP-compliant testing framework for Bash 3.2 or above. It provides a simple way to verify that the UNIX programs you write behave as expected."

Bats works "with any Unix program," not just Bash - it captures stdout / stderr / exit code of any executable.

When to use

• Testing a shell script (deployment scripts, CI scripts, installers).
• Testing a CLI binary's interface (commands, flags, exit codes, output) at the integration layer.
• Validating that wrapper scripts (e.g., kubectl plugins, git aliases) behave as expected.
• Cross-platform smoke tests for a CLI's install scripts.

How to use

1. Install bats (npm / Homebrew / git submodule); pin via submodule for CI determinism (Step 1).
2. Create test/<feature>.bats starting with #!/usr/bin/env bats and one @test block per behavior (Step 2).
3. Wrap each command under test with run, then assert on $status, $output, and $lines[] (Step 3).
4. Add setup / teardown with mktemp -d so each test is isolated and parallel-safe (Step 4).
5. load shared helpers plus bats-support / bats-assert for diff-style assertions (Steps 5-6).
6. Guard environment-specific tests with skip; run the suite via bats test/, adding --jobs N for parallelism (Steps 7-8).
7. Wire it into CI with TAP or JUnit output (references/ci-integration.md).

Step 1 - Install

Per bgh:

# npm
npm install -g bats

# Homebrew
brew install bats-core

# Git submodule (vendored, deterministic for CI)
git submodule add https://github.com/bats-core/bats-core.git test/bats
git submodule add https://github.com/bats-core/bats-support.git test/test_helper/bats-support
git submodule add https://github.com/bats-core/bats-assert.git test/test_helper/bats-assert

The git-submodule approach pins the Bats version per repo - recommended for CI determinism.

Step 2 - First test file

Per bgh (verbatim example):

#!/usr/bin/env bats

@test "addition using bc" {
  result="$(echo 2+2 | bc)"
  [ "$result" -eq 4 ]
}

@test "addition using dc" {
  result="$(echo 2 2+p | dc)"
  [ "$result" -eq 4 ]
}

Save as test/math.bats; run bats test/math.bats.

Step 3 - run for exit-code + output capture

The run helper invokes a command and captures $status, $output, and $lines[]:

@test "myscript --help exits 0 and mentions usage" {
  run ./myscript --help
  [ "$status" -eq 0 ]
  [[ "$output" == *"Usage:"* ]]
}

@test "myscript with no args exits 1" {
  run ./myscript
  [ "$status" -eq 1 ]
  [ "$output" = "error: missing required argument" ]
}

@test "myscript output line count" {
  run ./myscript list
  [ "$status" -eq 0 ]
  [ "${#lines[@]}" -eq 3 ]
  [ "${lines[0]}" = "alice" ]
}

Without run, a non-zero exit code aborts the test before assertions execute.

Step 4 - setup / teardown

Per bats: "setup and teardown: Pre- and post-test hooks."

setup() {
  TEST_DIR="$(mktemp -d)"
  cp fixtures/sample.txt "$TEST_DIR/"
}

teardown() {
  rm -rf "$TEST_DIR"
}

@test "myscript processes the sample file" {
  run ./myscript "$TEST_DIR/sample.txt"
  [ "$status" -eq 0 ]
}

setup_file / teardown_file run once per file (vs once per test). Use for expensive setup (Docker container start, database init).

Step 5 - load shared helpers

Per bats: "load: Share common code."

# test/test_helper.bash
make_temp_repo() {
  local d
  d="$(mktemp -d)"
  (cd "$d" && git init -q)
  echo "$d"
}

# test/integration.bats
load test_helper

@test "git status in fresh repo is clean" {
  local d
  d="$(make_temp_repo)"
  cd "$d"
  run git status --porcelain
  [ "$status" -eq 0 ]
  [ -z "$output" ]
}

Step 6 - bats-assert / bats-support

Better assertions than raw [ ]:

load test_helper/bats-support/load
load test_helper/bats-assert/load

@test "myscript produces expected JSON" {
  run ./myscript --json
  assert_success
  assert_output --partial '"version":'
  refute_output --partial 'ERROR'
}

@test "myscript fails on bad input" {
  run ./myscript --bad-flag
  assert_failure 2
  assert_line --index 0 "Unknown flag: --bad-flag"
}

assert_success / assert_failure / assert_output / assert_line give clear diff-style failure messages (vs [ ]'s silent fail).

Step 7 - skip

Per bats: "skip: Easily skip tests."

@test "macOS-only behavior" {
  if [[ "$(uname -s)" != "Darwin" ]]; then
    skip "macOS-only test"
  fi
  run ./myscript --use-keychain
  [ "$status" -eq 0 ]
}

@test "needs docker daemon" {
  if ! command -v docker >/dev/null; then
    skip "docker not installed"
  fi
  if ! docker info >/dev/null 2>&1; then
    skip "docker daemon not running"
  fi
  run ./deploy-script --dry-run
  [ "$status" -eq 0 ]
}

skip reports the reason in test output (vs commenting out, which hides the fact that coverage dropped).

Step 8 - Parallel execution

Per bats: "Parallel Execution."

# Install GNU parallel first (apt install parallel / brew install parallel)
bats --jobs 4 test/

Tests must be independent - share no global filesystem state, no shared ports. Use setup/teardown with mktemp -d to isolate.

Step 9 - Output formats and CI

bats emits pretty output by default, TAP or JUnit XML for CI, and runs in the official bats/bats Docker image. The output-format flags, a GitHub Actions job, and the Docker one-liner are in references/ci-integration.md.

Worked example

You maintain release.sh, a shell script that prints usage on --help and exits non-zero when called with no arguments. Cover both paths with Bats:

1. Vendor Bats as a submodule (Step 1), then create test/release.bats:

   #!/usr/bin/env bats
   
   load test_helper/bats-support/load
   load test_helper/bats-assert/load
   
   @test "release.sh --help exits 0 and prints usage" {
     run ./release.sh --help
     assert_success
     assert_output --partial "Usage:"
   }
   
   @test "release.sh with no args exits non-zero" {
     run ./release.sh
     assert_failure
     assert_output --partial "missing required argument"
   }
   

2. Run it: bats test/release.bats. Bats reports:

   release.bats
    ✓ release.sh --help exits 0 and prints usage
    ✓ release.sh with no args exits non-zero
   
   2 tests, 0 failures
   

3. In CI, swap to bats --formatter junit test/ > junit.xml and upload the report (see references/ci-integration.md).

Anti-patterns

Anti-pattern	Why it fails	Fix
Calling commands directly without run	Non-zero exit aborts the test before any assertion runs.	Wrap with run; check $status.
Tests sharing /tmp / fixed ports	Parallel execution breaks intermittently.	Per-test mktemp -d in setup (Step 4).
Using [ ] for everything	Silent failures; no diff message.	bats-assert (Step 6).
Commenting out skipped tests	Hides coverage loss; no signal.	skip with reason (Step 7).
Asserting full multiline output	Brittle - any whitespace change breaks tests.	Use assert_output --partial or assert_line --index N.
One mega .bats file	Slow; hard to grep failures.	Per-feature files in test/.

Limitations

• Bash-only. Tests are written in Bash; PowerShell scripts need Pester instead.
• Slow vs in-language tests. Each run forks a process - expensive vs library-level tests in the language.
• Output capture quirks. run strips trailing newlines; some binary output is not faithfully captured.
• Windows requires WSL or Git Bash. Per bats: "Installing Bats from source via Git Bash."

References

• bats - overview, install methods, test structure, @test / run / setup / teardown / load / skip, parallel execution, Docker.
• bgh - example test file, install one-liners.
• references/ci-integration.md - output formats, GitHub Actions job, Docker image.
• tui-snapshot-tester - TUI snapshot tests (visual layer; bats covers exit code + text output).
• cli-output-conventions - what to assert on (stable formats, exit codes, stderr).