manage-connector

Creating External Connectors

Use CREATE TABLE statements with WITH clause to connect external data sources and sinks:

CREATE TABLE MyTable (
    column1 TYPE,
    column2 TYPE,
    event_time TIMESTAMP_LTZ(3) NOT NULL METADATA FROM 'timestamp',
    WATERMARK FOR event_time AS event_time - INTERVAL '1' SECOND
) WITH (
    'connector' = 'connector-name',
    'option1' = 'value1',
    ...
);

Formatting rules — always follow, never compress onto a single line:

• CREATE TABLE name ( on its own line; each column definition on its own line with 2-space indent; ) WITH ( on its own line
• Each connector property goes on its own line inside WITH (, indented 4 spaces, with spaces around = (e.g. 'connector' = 'kafka-safe')
• Connector options that apply to only one runtime mode (streaming vs. batch) go inside conditional mustache blocks — {{#is_batch}} … {{/is_batch}} for batch-only options, {{^is_batch}} … {{/is_batch}} for streaming-only ones; {{#is_batch}} / {{/is_batch}} each go on their own line with no leading indent
• Separate each CREATE TABLE block with a blank line when multiple tables appear in the same file

Connector Sources

Before writing or editing a WITH (...) clause, every time, fully read the linked documentation of the connector you are about to use for the connector specific configuration options, even when you are copying a CREATE TABLE that already exists in the project.

• kafka and kafka-safe: Read or write append-only streams from Kafka-compatible data sources. Prefer the -safe version which adds DLQ and smart watermark support.
• upsert-kafka and upsert-kafka-safe: Read or write change streams for Kafka-compatible data sources. Prefer the -safe version which adds DLQ support.
• filesystem: Read or write data from local and cloud storage systems.
• iceberg: Read or write data from Apache Iceberg.
• jdbc: Write data via JDBC or use for lookup joins.
• datagen: Generate synthetic data for testing.
• print: Print output to stdout for debugging.
• blackhole: Discards all output (useful for testing).