data-pipelines

Data Pipelines Agent Skill

This skill specializes in creating animated data pipeline and ETL flow diagrams using the Excalimate MCP server.

When to Use

Use this skill when users ask to visualize:

• Data pipelines and ETL/ELT processes
• Streaming architectures (Kafka, RabbitMQ, SQS)
• Data warehousing flows
• ML pipelines and data science workflows
• Event streaming topologies
• Data transformation workflows
• Any scenario involving "data flow" or "how the data moves"

Workflow

1. Identify Components: Sources, transforms, sinks, queues, routers
2. Choose Topology: Linear pipeline, fan-out, fan-in, diamond, or streaming
3. Layout Left-to-Right: Sources → Processing → Sinks with 350px stage separation
4. Color-Code by Role: Green sources, blue transforms, purple sinks, etc.
5. Animate Left-to-Right: Stage-by-stage reveal from sources to sinks

Components

All components use consistent sizing and include boundElements for labels:

Data Source (Green)

{
  "type": "rectangle",
  "width": 160,
  "height": 70,
  "strokeColor": "#2f9e44",
  "backgroundColor": "#b2f2bb",
  "boundElements": [{"id": "label-id", "type": "text"}]
}

Transform (Blue)

{
  "type": "rectangle", 
  "width": 180,
  "height": 80,
  "strokeColor": "#1971c2",
  "backgroundColor": "#a5d8ff",
  "boundElements": [{"id": "label-id", "type": "text"}]
}

Sink (Purple)

{
  "type": "rectangle",
  "width": 160, 
  "height": 70,
  "strokeColor": "#6741d9",
  "backgroundColor": "#d0bfff",
  "boundElements": [{"id": "label-id", "type": "text"}]
}

Queue/Stream (Orange)

{
  "type": "rectangle",
  "width": 200,
  "height": 50, 
  "strokeColor": "#f08c00",
  "backgroundColor": "#ffec99",
  "boundElements": [{"id": "label-id", "type": "text"}]
}

Router (Teal Diamond)

{
  "type": "diamond",
  "width": 140,
  "height": 120,
  "strokeColor": "#0c8599", 
  "backgroundColor": "#99e9f2",
  "boundElements": [{"id": "label-id", "type": "text"}]
}