official-conference-paper-census

Official Conference Paper Census

Use this skill to turn an official conference listing into an auditable dataset. Keep two authorities separate:

• Acceptance authority: the conference website, official proceedings, official publisher, or official OpenReview conference record. This determines venue identity and the primary paper_url.
• Content evidence: the official PDF when available, otherwise an OpenAlex-resolved repository copy or a reviewed arXiv/institutional copy that is identity-matched to the official record. This supplies abstract/full-text bytes only and never proves acceptance.

An arXiv-only paper is never eligible for the main catalog.

Workflow

1. Freeze the current catalog and inspect the repository schema before changing data.
2. Capture the official conference entry point, fetched timestamp, track names, published count, and source URL.
3. Prefer an official machine-readable export. For ICLR, use scripts/export_iclr_downloads.py; the official ICLR Downloads 2026 form supports JSON/CSV/XLSX/YAML and independent track selection. Record the Downloads event total separately from the main-conference proceedings total because the page mixes posters, tutorials, invited talks, workshops, and demonstrations.
4. Check references/conference-support.md before choosing an adapter. Use the venue adapter already present in scripts/build_conference_census.py or add a small official-source adapter. It must preserve the conference's own track labels and use a first-party page, proceedings record, official publisher page, or official OpenReview record as the source of truth. For Researchr venues (ASE/FSE/ISSTA/ICSE/PLDI/POPL/OOPSLA), the accepted-paper page is the census and the official event-details AJAX response is the abstract source. For ICML, use the official poster page; for AAAI, use the official OJS article page. Store source hashes, response metadata, raw snapshots where practical, and an append-only JSONL ledger. Per-host concurrency and pacing must be bounded; challenge pages are recorded as pending and are never bypassed. If a first-party host fails only because the local HTTP proxy cannot complete its TLS connection, retry through a direct connection for that exact host (for example with a narrowly scoped NO_PROXY entry); this is transport troubleshooting, not a challenge or CAPTCHA bypass. Keep the official URL and the transport outcome in the audit ledger.
5. Normalize every official record into the census with its title, venue, year, track, official URL, full-text URL when available, and a unique key. Give every record a disposition: included, excluded, pending, or duplicate. Every non-included record needs a concrete reason.
6. Refresh Researchr track pages before doing per-paper requests: scripts/refresh_researchr_extensions.py merges newly exposed DOI/PDF identifiers without resetting prior review state. Then run scripts/enrich_official_pdf_urls.py for unresolved AAAI/ICML/Researchr/SIGPLAN candidates. ACM DOI records may be mapped to the official dl.acm.org PDF endpoint, but never bypass publisher challenges or bulk-download against host policy. “No PDF exposed” means only that no usable first-party link was visible at the recorded fetch time.
7. Run scripts/enrich_official_metadata.py first. When the official host omits metadata or blocks content, run scripts/enrich_scholarly_content.py --conference <ID>. It batches official DOIs through OpenAlex, reconstructs abstracts, records open locations, applies reviewed overrides from data/audit/content-source-overrides.yaml, and checkpoints every result in data/audit/2026-scholarly-content.jsonl. A DOI mismatch is rejected. A non-DOI override must bind to the exact official detail URL and record reviewed identity evidence. Prefer exact title plus authors; when a submitted manuscript and accepted paper have different titles, require a complete author match plus materially identical abstract, method, datasets, and reported results, and preserve the title-change caveat explicitly.
8. Run scripts/scan_conference_fulltext.py in batches. The default is metadata-first: an identity-bound title/abstract filters clearly unrelated records before any PDF request, while candidates enter the PDF queue. For publisher-constrained venues, use --verified-content-only so only resolved open copies are requested. Use --pdf-scope all only for an explicitly documented full-PDF pass. Store source identity, version, retrieval hash, extraction status, page/snippet evidence, and model candidates in data/audit/2026-fulltext-scan.jsonl; do not commit PDFs. Resume from the JSONL checkpoint.
9. Review every product hit in context. Search the full text, appendix, tables, footnotes, supplementary material, and artifact documentation for Claude Code, Claude-Code, Claude Code CLI, Codex CLI, Codex-CLI, Repo Codex, OpenAI Codex, Codex agent, and coding agent. A paper using a Claude or GPT API is not automatically a paper about Claude Code or Codex CLI.
10. Extract the paper's original model string and configuration. Preserve snapshots such as gpt-5.2-2025-12-11 verbatim; record not-reported only after the full text and available artifact have been checked. Save the section/page/table/appendix location for each claim.
11. Review recall-oriented product hits with scripts/review_iclr_product_hits.py or scripts/review_other_product_hits.py so references and ancillary author assistance become excluded records with reasons rather than silent omissions. If a venue has unreviewed product hits, stop promotion and add an explicit review mapping.
12. Before promotion, run scripts/update_pending_review.py. It identifies direct product signals in identity-bound titles/abstracts, records whether content is unresolved, challenged, or awaiting scanning, and writes data/audit/2026-pending-summary.json. A high-priority pending record is still pending; title/abstract evidence alone never qualifies it for the catalog.
13. Promote only records that have an exact official record and reviewed product-level evidence. Run scripts/finalize_catalog_audit.py, then generate docs/2026-conference-census.md with scripts/build_audit_report.py.