Borrowing it
Nothing to install: this file belongs to OpenUploading/CogniFold. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/OpenUploading/CogniFold/main/.claude/skills/longmemeval-run/SKILL.mdgit clone --depth 1 https://github.com/OpenUploading/CogniFoldWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/openuploading/cognifold/longmemeval-run)<a href="https://agentmods.dev/skills/openuploading/cognifold/longmemeval-run"><img src="https://agentmods.dev/badge/skills/openuploading/cognifold/longmemeval-run/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/openuploading/cognifold/longmemeval-run"><img src="https://agentmods.dev/badge/skills/openuploading/cognifold/longmemeval-run.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00130 | $0.01668 |
| Opus 5.5 | $0.00052 | $0.00667 |
| Sonnet 5.5 | $0.00026 | $0.00334 |
| Haiku 4.5 | $0.00013 | $0.00167 |
Grade A, and why
longmemeval-run scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
`curl` / `python` invocation in chat. How it starts
The opening of the file, as written. The whole thing — 126 lines — stays where its author put it; the contents beside it link to each section on GitHub.
LongMemEval One-Shot Run
When to invoke
- User just did
git cloneand asks "how do I run LongMemEval" - User says "run the benchmark" / "open-box test" / "first run"
- Previous run failed and the user wants to retry from scratch
- A new chat / embed / judge provider was added — verify each endpoint then re-run the full benchmark
- After
git pullbrings new code — verify and re-run
Do NOT invoke if the user is asking to iterate on the score (that's
longmemeval-iterate) or to run another benchmark (LoCoMo, etc.).
Recommended stack (defaults)
These are the models the script pings at steps 6-8 and then uses in the
full N=500 run. Defaults live in scripts/parallel_longmemeval.sh
(source of truth) and are mirrored in scripts/run.sh for the ping
checks.
| Role | Default | Why |
|---|---|---|
| reader | openai:gpt-5 (reasoning_effort=high auto) |
the qa_answer prompt assumes a reasoning model — strongest available reader |
| writer | openai:gpt-5 (reasoning_effort honored from env) |
strongest extractor — preserves attributes the user named verbatim |
| judge | openai:gpt-4o |
canonical LongMemEval judge; do not substitute without re-calibrating against published numbers |
| rerank | openai:gpt-5 (batched, pool=100) |
one batched call per question; handles "27th item" / ordinal references |
| embed | openai:text-embedding-3-large (1536 dim via API dimensions param) |
strongest embedding; cognifold/embeddings/providers.py passes dimensions=1536 so the graph schema stays compatible |
Override any role by exporting READER_MODEL / WRITER_MODEL /
JUDGE_MODEL / RERANK_MODEL / EMBED_MODEL before invoking the
script.
Stratified sampling: 133 per question type (one type, temporal- reasoning, has 133 questions — the cap means "take all" for every
type, so the full 500 set is processed). Override with the second
positional arg to scripts/parallel_longmemeval.sh if needed.
W1 (EXTRACT_TYPED_ATTRIBUTES=1) is enabled by default in run.sh.
W2 (RESOLVE_EVENT_DATES=1) is off by default.
What ships with it
1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2mo ago First seen · 126 lines · 130 tokens per session scan A 84a4c086655e
longmemeval-run is a skill published in the GitHub repository OpenUploading/CogniFold (63 stars, last pushed 2mo ago), licensed Apache-2.0. It adds 130 tokens to every session and 1,668 once invoked, about $0.0005 per session on Opus 5.5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
dev-browser
Browser automation with persistent page state. Use when users ask to navigate websites, fill forms, take screenshots, extract web data, test web apps, or automate browser workflows. Trigger phrases include "go to [url]", "click on", "fill out the form", "take a screenshot", "scrape", "automate", "test the website"…
project-knowledge
Verified, actionable gotchas for oh-my-posh. Read the matching topic file BEFORE changing shell integration scripts (pwsh, zsh, bash, fish, cmd/Clink, nu, elvish, xonsh), terminal or pty behavior, WSL or e2e shell tests, Go internals (cache, segments, streaming, serve daemon, template rendering, output sanitization)…
potpie-debug-memory
Use while debugging or troubleshooting failures, flaky tests, incidents, production alerts, CI failures, local dev setup issues, repeated bugs, prior fixes, failed attempts, and verification history.
impl-validator
Independently check whether an implementation matches its stated goal and return PASS/WARN/FAIL with actionable issues. Use for second-opinion validation of an implementation or generated output.
self-improve-with-tinyfish
Enables Hermes to create new reusable skills for itself by researching live web sources with TinyFish Search and Fetch, analyzing source coverage, writing SKILL.md files, and installing them into Hermes memory. Use when the user asks Hermes to learn, teach itself, upgrade itself, or save a reusable capability.
performing-fuzzing-with-aflplusplus
Perform coverage-guided fuzzing of compiled binaries using AFL++ (American Fuzzy Lop Plus Plus) to discover memory corruption, crashes, and security vulnerabilities. The tester instruments target binaries with afl-cc/afl-clang-fast, manages input corpora with afl-cmin and afl-tmin, runs parallel fuzzing campaigns with…