running-julia-test

Running Julia tests

Running all tests with Pkg.test()

Before running tests for a new package, make sure the test environment is already set up. If the package does not have tests yet, follow creating-julia-test-env first. Tests that use using Test must have Test available from the package or test project.

To run all of your package's tests, use the following command:

$ julia --project -e 'using Pkg; Pkg.test()'

Running specific test sets only

To run only particular test sets (for example, those defined as @testset "label" begin ... end), use the ~/.julia/bin/testrunner tool.

If testrunner is already installed, this command prints its path:

$ command -v testrunner

If you do not have the testrunner command, install it with:

$ julia -e 'using Pkg; Pkg.activate(); Pkg.Apps.add(url="https://github.com/aviatesk/TestRunner.jl")'

Basic Usage

$ testrunner demo.jl "basic tests"                   # Run the testset named "basic tests"

$ testrunner demo.jl "basic tests" "struct tests"    # Run multiple named testsets

$ testrunner demo.jl '(:(@test startswith(inner_func2(), "inner")))'  # Run a standalone test case

See the guide to learn more.

Caveat: There are cases where name or regex filters do not work as expected

Sometimes, even after specifying a testset name or a regex, all tests may run instead of just the intended ones (this seems to be due to a pattern matching behavior in TestRunner.jl). Example:

# Expected: Only "calculate: basic operations" should run (4 tests), but actually all 20 tests run
$ testrunner --project=. test/runtests.jl "calculate: basic operations"
$ testrunner --project=. test/runtests.jl 'r"^calculate: basic operations$"'
$ testrunner --project=. test/runtests.jl L6:11   # Even if you include the @testset declaration line, all tests are executed

As a workaround, specify the line range including only the @test lines (do not include the @testset ... begin declaration line). This way, you can ensure that only the intended testset is executed: