Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add vaquarkhan/data-engineering-agent-skills --skill airflow-and-workflow-orchestrationgit clone --depth 1 https://github.com/vaquarkhan/data-engineering-agent-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/vaquarkhan/data-engineering-agent-skills/airflow-and-workflow-orchestration)<a href="https://agentmods.dev/skills/vaquarkhan/data-engineering-agent-skills/airflow-and-workflow-orchestration"><img src="https://agentmods.dev/badge/skills/vaquarkhan/data-engineering-agent-skills/airflow-and-workflow-orchestration/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/vaquarkhan/data-engineering-agent-skills/airflow-and-workflow-orchestration"><img src="https://agentmods.dev/badge/skills/vaquarkhan/data-engineering-agent-skills/airflow-and-workflow-orchestration.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00061 | $0.00897 |
| Opus 5 | $0.00030 | $0.00449 |
| Sonnet 5 | $0.00012 | $0.00179 |
| Haiku 4.5 | $0.00006 | $0.00090 |
Grade A, and why
airflow-and-workflow-orchestration scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 12d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 89 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Airflow And Workflow Orchestration
Overview
Use this skill when the problem is orchestration rather than transformation logic. It helps agents design Airflow, MWAA, Cloud Composer, Azure Data Factory, Step Functions, Google Cloud Workflows, or Databricks Workflows patterns with explicit dependencies, retries, ownership, backfills, and publish-safe cutover behavior.
When to Use
- building or modifying
AirflowDAGs - choosing between scheduler-driven, event-driven, or platform-native orchestration
- designing scheduling, sensors, task dependencies, or retry policy
- coordinating ingestion, transformation, quality checks, and publish steps
- changing backfill, catchup, or SLA behavior
Do not use this as a substitute for the underlying processing skill. Orchestration coordinates work; it does not define the compute logic itself.
Workflow
-
Define the workflow contract. Capture:
- owner
- schedule or trigger mode
- upstream and downstream dependencies
- task boundaries
- success and failure signals
-
Choose the orchestration model before choosing the product. Decide whether the workload is:
- scheduler-driven
- event-driven
- metadata-driven fan-out
- platform-native inside a lakehouse
-
Separate orchestration concerns from processing concerns. Workflow tasks should call well-defined jobs, not hide business logic in orchestration code.
-
Design retries and timeouts deliberately. Account for:
- idempotency
- duplicate writes
- sensor cost
- late-arriving upstream data
- alert routing
-
Make backfill behavior explicit. Decide how catchup, reruns, and historical windows behave before enabling them.
-
Gate publish steps on validation. A successful task chain is not enough if downstream tables fail quality checks.
Service Selection Hints
AWS: preferMWAAfor dependency-rich DAGs,Step Functionsfor branching and service coordination, and lightweight event triggers for arrival-based flows.Azure: preferAzure Data FactoryorSynapse Pipelinesfor connector-heavy and parameterized workflows, andDatabricks Workflowswhen execution stays inDatabricks.GCP: preferCloud Composerfor DAGs,Google Cloud Workflowsfor API and service coordination, andCloud SchedulerplusPub/Subfor simple triggers.Databricks: preferDatabricks WorkflowsorDelta Live Tablesfor platform-native jobs, and use an external orchestrator when dependencies cross platform boundaries.- Load
references/pipeline-orchestration-patterns.mdwhen selecting or reviewing the orchestration model, not only the syntax of one tool.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 12d ago First seen · 89 lines · 61 tokens per session scan A 9829eb371c3e
airflow-and-workflow-orchestration is a skill published in the GitHub repository vaquarkhan/data-engineering-agent-skills (45 stars, last pushed 3mo ago), licensed MIT. It adds 61 tokens to every session and 897 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
cost-optimization
Optimize cloud costs across AWS, Azure, GCP, and OCI through resource rightsizing, tagging strategies, reserved instances, and spending analysis. Use when reducing cloud expenses, analyzing infrastructure costs, or implementing cost governance policies.
terraform-module-library
Build reusable Terraform modules for AWS, Azure, GCP, and OCI infrastructure following infrastructure-as-code best practices. Use when creating infrastructure modules, standardizing cloud provisioning, or implementing reusable IaC components.
linkerd-patterns
Implement Linkerd service mesh patterns for lightweight, security-focused service mesh deployments. Use when setting up Linkerd, configuring traffic policies, or implementing zero-trust networking with minimal overhead.
dokploy-deploy
This skill should be used when user asks to "deploy with Dokploy", "use Dokploy Cloud", "manage self-hosted Dokploy", "deploy Docker Compose on Dokploy", "manage Dokploy databases", "configure Dokploy domains", or "look up Dokploy CLI commands".
gcloud-usage
This skill should be used when user asks about "GCloud logs", "Cloud Logging queries", "Google Cloud metrics", "GCP observability", "trace analysis", or "debugging production issues on GCP".
openship-config
This skill should be used when the user asks to "create openship.json", "configure an OpenShip deployment", "make a repo deployable on OpenShip", or fix "openship config validate" errors.