monke

Monke - Airweave Connector Testing Framework

End-to-end testing framework for Airweave connectors using real external APIs

Overview

Monke validates Airweave's data synchronization pipeline by creating real test data in external systems, triggering sync jobs, verifying data in the vector database, and testing update/deletion scenarios. The framework separates configuration from runtime state, uses pluggable authentication, and supports parallel execution.

---

Core Architecture

Test Runner (runner.py) - Entry point that orchestrates parallel test execution across connectors

Test Flow (core/flow.py) - Executes complete lifecycle for a connector: setup → test steps → cleanup

Test Steps (core/steps.py) - Individual operations (create, sync, verify, update, delete) with retry logic

Configuration (core/config.py) - YAML-based config with classes: TestConfig, ConnectorConfig, TestFlowConfig, DeletionConfig

Test Context (core/context.py) - Runtime state separate from config: tracks entities, infrastructure IDs, metrics

Infrastructure (core/infrastructure.py) - Creates/tears down test collections and source connections

Authentication (auth/) - Pluggable auth brokers (BaseAuthBroker, ComposioBroker) for credential resolution

---

Test Flow

Default test steps execute in sequence:

1. collection_cleanup       # Clean leftover test collections
2. cleanup                  # Clean entities in external system
3. create                   # Create test entities
4. sync                     # Trigger Airweave sync
5. verify                   # Verify entities in vector DB
6. update                   # Update entities
7. sync → verify            # Verify updates
8. partial_delete           # Delete subset of entities
9. sync → verify_partial_deletion
10. verify_remaining_entities
11. complete_delete         # Delete all entities
12. sync → verify_complete_deletion
13. cleanup                 # Final cleanup
14. collection_cleanup      # Delete test collection