triaging-security-alerts-in-splunk

Triaging Security Alerts in Splunk

When to Use

Use this skill when:

• SOC Tier 1 analysts need to process the Incident Review queue in Splunk Enterprise Security (ES)
• Notable events require rapid severity classification and initial investigation before escalation
• Alert volume exceeds capacity and analysts need a systematic triage methodology
• Management requests metrics on alert disposition (true positive, false positive, benign)

Do not use for deep forensic investigation — escalate to Tier 2/3 after initial triage confirms malicious activity.

Prerequisites

• Splunk Enterprise Security 7.x+ with Incident Review dashboard configured
• CIM-normalized data sources (Windows Event Logs, firewall, proxy, endpoint)
• Role with ess_analyst capability for notable event status updates
• Familiarity with SPL (Search Processing Language)

Workflow

Step 1: Access Incident Review and Prioritize Queue

Open the Incident Review dashboard in Splunk ES. Sort notable events by urgency (calculated from severity x priority). Apply filters to focus on unassigned events:

| `notable`
| search status="new" OR status="unassigned"
| sort - urgency
| table _time, rule_name, src, dest, user, urgency, status
| head 50

Focus on Critical and High urgency events first. Group related alerts by src or dest to identify attack chains rather than treating each alert independently.

Step 2: Investigate the Notable Event Context

For each notable event, pivot to raw events. Example for a brute force alert:

index=wineventlog sourcetype="WinEventLog:Security" EventCode=4625
src_ip="192.168.1.105"
earliest=-1h latest=now
| stats count by src_ip, dest, user, status
| where count > 10
| sort - count

Check if the source IP is internal (lateral movement) or external (perimeter attack). Cross-reference with asset and identity lookups:

| `notable`
| search rule_name="Brute Force Access Behavior Detected"
| lookup asset_lookup_by_cidr ip AS src OUTPUT category, owner, priority
| lookup identity_lookup_expanded identity AS user OUTPUT department, managedBy
| table _time, src, dest, user, category, owner, department