reviewer-defense

Reviewer Defense Methodology

You are helping a researcher prepare for peer review by identifying weaknesses, selecting the strongest results, and drafting responses to likely questions.

Step 1: Vulnerability Analysis

Read the paper and identify weaknesses from a reviewer's perspective:

Technical Weaknesses

• Missing baselines that reviewers would expect
• Evaluation metrics that don't fully capture the contribution
• Assumptions stated without justification
• Scalability concerns not addressed
• Missing error analysis or failure case discussion

Presentation Weaknesses

• Claims stronger than evidence supports
• Missing related work that a reviewer in the area would know
• Unclear methodology (could someone reimplement from the paper alone?)
• Figures that don't clearly convey the intended message
• Inconsistencies between sections

Experimental Weaknesses

• Small dataset size without justification
• Missing statistical significance tests
• No comparison with state-of-the-art on standard benchmarks
• Hyperparameter sensitivity not explored
• No computational cost comparison

Step 2: Venue-Specific Anticipation

Different venues have different review cultures:

Top-tier ML/CV conferences (CVPR, NeurIPS, ICLR, ECCV):

• Expect extensive ablation studies
• Strong baseline comparisons required
• Novelty must be clearly articulated
• Reproducibility is valued

Workshops:

• More tolerant of work-in-progress
• Interesting ideas valued over exhaustive evaluation
• Novel applications of existing methods are acceptable

Journals:

• Expect thorough related work discussion
• Deeper analysis and more experiments than conferences
• Writing quality and organization matter more

Step 3: Question Generation

Generate likely reviewer questions, ranked by probability:

For each question:

1. The question — phrased as a reviewer would write it
2. Why they'd ask — what triggers this concern
3. Can existing data answer it? — yes (point to specific data) or no (new experiment needed)
4. Draft response — if answerable, write a concise response