dt-alerting

dt-alerting

Configure and understand the full alerting lifecycle in Dynatrace — from anomaly detector setup through Grail event storage, problem grouping, and workflow notification delivery.

The Alerting Lifecycle

┌─────────────────────────────────────────────────────────────────────┐
│  Alert Sources — five categories, each fires a DAVIS_EVENT          │
│  ─────────────────────────────────────────────────────────────────  │
│  1. DQL-based  · Grail-scheduled server-side detector               │
│  2. Edge       · OneAgent on monitored host or process              │
│  3. Pipeline   · OpenPipeline ingest-stream filter matcher          │
│  4. Synthetic  · Worldwide synthetic checker node                   │
│  5. External   · Events API, Workflow, or OneAgent local ingest     │
└──────────────────────────────┬──────────────────────────────────────┘
                               │ DAVIS_EVENT created per trigger per entity
                               ▼
             ┌─────────────────────────────────────┐
             │  Event stored in Grail              │  Persisted and queryable via DQL.
             └──────────────────┬──────────────────┘  One event per trigger per entity.
                                │ correlated by root-cause and impact graph
                                ▼
             ┌─────────────────────────────────────┐
             │  Problem (Denoising)                │  Events sharing the same root-cause
             └──────────────────┬──────────────────┘  and impact graph → one Problem.
                                │ problem event triggers workflow
                                ▼
             ┌─────────────────────────────────────┐
             │  Workflow Notification              │  Filters problems and routes to
             └─────────────────────────────────────┘  email, Slack, ServiceNow, webhook.

When to Use This Skill

• Detector setup — "How do I create an anomaly detector?", "What kind of detector should I use?", "What is the difference between adaptive and seasonal?"
• Alert event history — "Query all alert events for this service", "Show me which metrics triggered alerts last week"
• Problem denoising — "Why did these two alerts merge into one problem?", "How does Davis group alerts?"
• Notification setup — "How do I send a Slack message when a problem opens?", "Set up a ServiceNow ticket on critical problems"
• Best practices — "How do I avoid alert storms?", "Which sensitivity setting should I use?"
• Over-alerting analysis — "Why am I getting too many alerts?", "How do I reduce alert fatigue?", "Which detector is firing the most?", "How do I tune sensitivity or thresholds to avoid noise?"
• Notification routing — "How do I route alerts to the right team?", "Set up scalable problem filters in workflows", "Send Slack notifications only to the team responsible for the affected service"