connector-development-end-to-end

Building and Testing a Source Connector: End-to-End Guide

Overview

This is the master guide for building a complete, production-ready source connector for Airweave. It combines source implementation with comprehensive E2E testing using the Monke framework.

Use this guide with your AI coding assistant to build connectors systematically.

---

Prerequisites

Note: The human has already completed these setup steps:

• ✅ OAuth credentials configured in backend/airweave/platform/auth/yaml/dev.integrations.yaml
• ✅ Monke authentication configured in monke/configs/{short_name}.yaml (Composio or direct)
• ✅ API documentation loaded into context

Your task is to write the code. The human will handle testing and running commands.

---

Important Guidelines

These are the most common mistakes when building connectors:

1. Make Entities Information-Rich (Embeddable Fields)

Rule: Mark ~70% of entity fields as embeddable=True

Why: Without embeddable=True, fields are only keyword-searchable, not semantically searchable. Users won't be able to find relevant data.

What to mark embeddable:

• ✅ All text content (descriptions, notes, comments, body)
• ✅ All names and titles
• ✅ All people (assignees, authors, owners, members)
• ✅ All status/metadata (status, priority, tags, labels)
• ✅ All timestamps (created_at, modified_at, due_dates)

What NOT to mark embeddable:

• ❌ Internal IDs (entity_id, external_id, database IDs)
• ❌ Binary metadata (sizes, checksums, mime_types)

Bad Example:

# Avoid: Sparse entity - users can't search by anything except name
class TaskEntity(ChunkEntity):
    name: str = AirweaveField(..., embeddable=True)
    description: str = Field(...)  # Should be embeddable
    assignee: Dict = Field(...)     # Should be embeddable

Good Example:

# Better: Information-rich - users can search everything
class TaskEntity(ChunkEntity):
    name: str = AirweaveField(..., embeddable=True)
    description: str = AirweaveField(..., embeddable=True)
    assignee: Dict = AirweaveField(..., embeddable=True)
    status: str = AirweaveField(..., embeddable=True)
    external_id: str = Field(...)  # ID correctly not embeddable