gx-review-test-types

Galaxy Test Type Review Command

Perform a code review on the provided Galaxy code. Accept input in any of following forms:

• A working directory path (analyze git diff in that directory)
• A Git commit reference (analyze changes in that commit)
• A PR reference (analyze changes in that pull request)
• A list of Python file paths (analyze those files)
• A planning document (analyze the Python files in the plan)

Primary Focus: Test Type Verification

The most common mistake is placing tests in the wrong location. Review each test file and verify it belongs in its current location.

Galaxy Test Types Decision Tree

Does test need running Galaxy server?
├─ NO → Unit test (test/unit/ or doctests)
└─ YES → Does test need web browser?
         ├─ NO → Does test need custom Galaxy config?
         │       ├─ NO → Is it testing only tools or workflows?
         │       │       ├─ Tools → Framework test (test/functional/tools/)
         │       │       ├─ Workflows → Workflow Framework test (lib/galaxy_test/workflow/)
         │       │       └─ Neither → API test (lib/galaxy_test/api/)
         │       └─ YES → Integration test (test/integration/)
         └─ YES → Does test need custom config?
                  ├─ NO → Selenium test (lib/galaxy_test/selenium/)
                  └─ YES → Selenium Integration test (test/integration_selenium/)

API vs Integration Test Distinction

API Tests (lib/galaxy_test/api/)

Use when ALL of these are true:

• Test needs running Galaxy server
• Test does NOT need custom Galaxy configuration
• Test does NOT need direct access to Galaxy internals (self._app, database models)
• Test can theoretically run against any external Galaxy server

Characteristics:

• Inherits from ApiTestCase
• Uses populators: DatasetPopulator, WorkflowPopulator, DatasetCollectionPopulator
• Uses HTTP methods: self._get(), self._post(), self._put(), self._delete()
• Tests via the Galaxy API only

Integration Tests (test/integration/)

Use when ANY of these are true:

• Test requires custom Galaxy configuration (enabled via handle_galaxy_config_kwds)
• Test requires Galaxy features that are off by default (quotas, specific object stores, job runners)
• Test requires direct access to self._app or database models
• Test requires external services started by the test (Docker containers, etc.)
• Test sets config options like: enable_quotas, job_config_file, object_store_config_file, metadata_strategy, ftp_upload_dir, custom auth settings