arxiv-paper-search

Arxiv Paper Search (Real-Time Knowledge)

Context

Static skills freeze at knowledge-cutoff date. ML papers come out daily. This skill enables the agent to fetch fresh academic knowledge directly from arxiv and Semantic Scholar at decision time. Essential for staying current with rapidly evolving fields (LLM agents, time series, multi-modal learning).

The core insight: skills + live search = always-current knowledge. Don't memorize; learn how to fetch.

Guidance

Basic Usage

from framework.src.knowledge import ArxivSearch

search = ArxivSearch()  # respects arxiv's 3s rate limit
papers = search.search("time series forecasting transformer", max_results=10)

for p in papers:
    print(f"[{p.year}] {p.title}")
    print(f"  Authors: {', '.join(p.authors[:3])}")
    print(f"  PDF: {p.pdf_url}")

Recent Papers by Category

# "What came out in cs.LG this week?"
recent = search.search_recent(category="cs.LG", days=7, max_results=20)

Use Semantic Scholar for Citations & TLDR

from framework.src.knowledge import SemanticScholar

ss = SemanticScholar()  # add api_key= for higher rate limits
papers = ss.search("tabular foundation models", max_results=10)

for p in papers:
    if p.tldr:
        print(f"TLDR: {p.tldr}")  # auto-generated 1-sentence summary
    print(f"Cited by: {p.citation_count} (influential: {p.influential_citation_count})")

Aggregate Everything

from framework.src.knowledge import KnowledgeAggregator

agg = KnowledgeAggregator()
report = agg.search_all("cross-competition feature transfer", max_per_source=10)
print(report.to_markdown())  # papers + kaggle discussions combined

# Optionally persist to vault for future recall
path = agg.write_report_to_vault(report)  # writes to docs/ml-agent-memory/auto-search/

Decision Workflow

1. Start a new technique exploration?
   → search_arxiv_recent(category="cs.LG", days=14)
2. Need to compare approaches?
   → search_papers(query) — Semantic Scholar returns TLDR + citations
3. Found an interesting one? Check follow-ups:
   → get_paper_citations(arxiv_id) — who built on it?
4. Combine with Kaggle discussions:
   → knowledge_search(query) — one-stop papers + forum
5. Persist findings:
   → write_report_to_vault(report)
   → mem.remember(MemoryItem(importance=0.8, type="experiment"))