Huntable-CTI-Studio: Skill for Claude Code

Add Source

This skill adds a new CTI intelligence source to config/sources.yaml and syncs it to the database. It handles both RSS-based and scraping-only sources.

Workflow

Step 1 — Gather source information

Collect from the user (ask if not provided):

Field	Required	Example
Blog/site URL	Yes	https://www.vmray.com/blog/
RSS feed URL	No	https://www.vmray.com/blog/feed/
Source name	Yes (can derive)	VMRay Blog

Step 2 — Discover source details

Use web tools to visit the site and determine:

1. RSS feed discovery — Check for <link rel="alternate" type="application/rss+xml"> in the page head, or try common paths (/feed/, /rss/, /feed.xml, /rss.xml, /atom.xml). If an RSS URL was provided, validate it returns valid XML.
2. Domain — Extract the domain for the allow list (e.g., vmray.com)
3. Post URL pattern — Look at article links to derive a post_url_regex (e.g., ^https://www\\.vmray\\.com/blog/.*)
4. Content selectors — Inspect an article page for appropriate body_selectors, title_selectors, date_selectors, author_selectors if the site needs scraping
5. Content type — Determine if articles are CTI-relevant (threat intel, malware analysis, detection engineering, vulnerability research)

Step 3 — Generate the source identifier

Create a snake_case id from the source name:

• VMRay Blog → vmray_blog
• Unit 42 Threat Research → unit42_threat_research

Check config/sources.yaml for duplicate identifiers before proceeding.

Step 4 — Build the YAML entry

Use this template, adapting based on discovery:

# {Source Name} — {brief note about RSS availability}
- id: "{identifier}"
  name: "{Source Name}"
  url: "{site_url}"
  rss_url: {rss_url or null}
  check_frequency: 14400  # 4 hours (system default; reduce to 1800 only after successful validation)
  active: false  # Start disabled; enable manually after verifying articles are ingested correctly
  config:
    allow: ["{domain}"]
    post_url_regex: ["{regex_pattern}"]
    robots:
      enabled: true
      user_agent: "Huntable-CTI-Studio/1.0 (+https://github.com/dfirtnt/Huntable-CTI-Studio)"
      respect_delay: true
      max_requests_per_minute: 10
      crawl_delay: 1.0
    min_content_length: 1500
    title_filter_keywords: ["webinar", "training", "careers", "job posting"]
    rss_only: {true if rss_url and no scraping needed, else false}
    extract:
      prefer_jsonld: true
      title_selectors: ["h1", "meta[property='og:title']::attr(content)"]
      date_selectors:
        - "meta[property='article:published_time']::attr(content)"
        - "time[datetime]::attr(datetime)"
      body_selectors: {discovered selectors or defaults}
      author_selectors: ["meta[name='author']::attr(content)", ".author-name"]
  description: "{one-line description of what intelligence this source provides}"