paper-filter

Paper Filter Skill

Filter CNS papers for machine learning task extraction suitability.

Input Requirements

Before invoking this skill, provide:

1. Paper Folder Path: Directory containing original paper files and preprocessed data
2. Output Directory: Directory to store filtering results

Paper folder structure:

• {paper_id}.pdf: Original paper PDF file
• {paper_id}.html: HTML version of the paper
• preprocessed/: Preprocessed data subdirectory, containing:
  • text.md: Full paper text
  • figures/: Figures directory
  • tables/: Tables directory
  • links.json: List of links from the paper (with section identifiers and surrounding context)

Supplementary Materials

preprocessed/text.md may not include supplementary content. When the main text references supplementary materials (tables, figures, or supplementary notes/text) for core results, metric details, evaluation protocols, or experimental details, and the information is not present in preprocessed/text.md:

1. Check preprocessed/links.json for supplementary material links (section: supplementary_information)
2. Download supplementary PDFs/files to a temporary location
3. Use the original paper PDF/HTML as fallback if supplementary links are not separately available
4. Extract relevant information (score tables, metric details, evaluation protocols, experimental details, and dataset descriptions)
5. Delete downloaded supplementary files after extraction is complete

Workflow

Phase 1: Understand Core Definitions

Read references/core_definitions.md to understand the basic criteria and requirements for task extraction.

Core definitions describe the structure of the candidate task tuple T = (A, Data, M, S, B):

• A (Algorithm): The core algorithm or strategy proposed by the paper
• Data: The data environment involved in the task, divided into D_dev (development data) and D_eval (evaluation data)
• M (Metric): The metric function used to measure result quality
• S (SOTA): The performance values of the proposed core algorithm on evaluation metrics
• B (Baseline): The baseline metric values reported in the paper (optional, may not exist)