buildkite-plugin-testing

Buildkite plugin testing

Plugins are shell, so they're tested with Bats (Bash Automated Testing System) run inside the buildkite/plugin-tester image, linted with buildkite/plugin-linter, and statically checked with shellcheck.

Verify current image tags / behavior via WebSearch on buildkite.com:

• Writing/testing plugins: https://buildkite.com/docs/pipelines/integrations/plugins/writing
• Plugin Tester: https://buildkite.com/buildkite/plugin-tester
• Bats: https://github.com/bats-core/bats-core

The three checks every plugin needs

1. shellcheck every hook script (catches unquoted vars, bad tests, etc.).
2. plugin-linter validates plugin.yml + README against your plugin id.
3. Bats exercises hook behavior, including failure paths.

Running locally

The tester image mounts your plugin at /plugin and runs bats tests/ by default:

docker run --rm -v "$PWD:/plugin:ro" buildkite/plugin-tester:latest

Or via docker-compose.yml (preferred, keeps lint + tests together):

services:
  tests:
    image: buildkite/plugin-tester:latest
    volumes: [".:/plugin:ro"]
  lint:
    image: buildkite/plugin-linter:latest
    command: ["--id", "your-org/your-plugin"]
    volumes: [".:/plugin:ro"]

docker-compose run --rm tests
docker-compose run --rm lint

Writing good Bats tests

Bats basics:

@test "description" {
  run "$PWD/hooks/command"      # captures $status and $output
  [ "$status" -eq 0 ]
  [ "$output" = "expected" ]
  [[ "$output" == *"substring"* ]]
}

Rules for plugin hook tests:

• Set the exact env vars the hook reads (BUILDKITE_PLUGIN_<NAME>_<KEY>). Getting the names right is most of the battle — see the buildkite-plugin-dev reference/env-var-mapping.md.
• Test the failure paths: missing required settings, invalid values. Assert both the non-zero $status and the error message.
• Cover array settings (_0, _1) and defaults (var unset).
• Don't call the network or real tools. Stub them (see below).
• Use setup() / teardown() for per-test fixtures and cleanup.