dagster-connections-to-orchestra

Dagster Resources -> Orchestra Connections

Overview

Dagster stores credentials in resources — typically ConfigurableResource subclasses (or built-in integration resources like SnowflakeResource, S3Resource, DbtCliResource) wired into Definitions(resources={...}), with secret values supplied via EnvVar. Orchestra connections serve the same purpose but live outside the pipeline code — set up in the Orchestra UI (Settings -> Connections), or through the public API / Orchestra MCP — and are referenced by name in pipeline YAML. This skill maps common Dagster resources to their Orchestra equivalents and covers naming, environment patterns, and secrets.

---

Connection Name Format

connection: my_snowflake_12345   # format: descriptive-name_XXXXX (5-digit suffix from UI)

The 5-digit suffix is assigned by Orchestra when the connection is created — never invent it. Read it back rather than asking someone to find it: connections are managed through the public API and the Orchestra MCP server as well as the UI, so listing the workspace's connections returns each one's descriptive_name_NNNNN id directly. Fall back to copying it from the UI only where neither is available.

No resource referenced? If the @op/@asset doesn't actually instantiate a credentialed resource — pure computation, no external client, no secrets — don't invent a connection name or a fake env var placeholder just to fill the field:

connection: null   # no distinct resource in the source; Orchestra uses the workspace default for this integration

Only set a specific name_XXXXX or ${{ ENV.VAR }} when the source code actually references a distinct resource/credential.

This extends to task parameters that duplicate connection-level scope, too — e.g. Power BI's workspace_id, or any other parameter whose value is also stored on the Orchestra connection itself. If the source code just reads the same single value everywhere (one env var, one resource-level config field) rather than genuinely varying it per task, leave that parameter null/omitted and let the connection's own configured value apply. Only carry an explicit value through (literal, input, or ${{ ENV.VAR }}) when a specific task truly needs to override it — e.g. targeting a different Power BI workspace than the one configured on the connection.