CONTEXTSCOUT_INTEGRATION_TESTS

A test suite for checking how OpenAgent, OpenCoder, and ContextScout work together to find project context. It includes scenarios for known information, unfamiliar areas, correct file selection, proactive discovery, and multiple domains.

In plain words
What is it for?
Running and reviewing integration tests for context discovery, including expected tool use, file accuracy, and coverage across different project areas.
Why use it?
It reveals whether agents choose the right discovery method and locate the right files when they do not already know the codebase. This makes integration problems easier to find before release.

Agent

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/darrenhinde/openagentscontrol/contextscout_integration_tests
Clone the repo
git clone --depth 1 https://github.com/darrenhinde/OpenAgentsControl
Per session 0 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 2,527 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00000 $0.02527
Opus 5 $0.00000 $0.01264
Sonnet 5 $0.00000 $0.00505
Haiku 4.5 $0.00000 $0.00253

Measured 3d ago against content hash b119bd9eb812, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

CONTEXTSCOUT_INTEGRATION_TESTS scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 3d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

evals/agents/CONTEXTSCOUT_INTEGRATION_TESTS.md · 316 lines

How it starts

The opening of the file, as written. The whole thing — 316 lines — stays where its author put it; the contents beside it link to each section on GitHub.

ContextScout Integration Test Suite

Purpose: Comprehensive validation that OpenAgent, OpenCoder, and ContextScout work together effectively for intelligent context discovery.

Created: 2026-01-09
Status: Ready to Run


Overview

This test suite answers the critical question: Should agents use ContextScout for context discovery, and if so, when and how?

What We're Testing

  1. OpenAgent Integration - Does OpenAgent use ContextScout proactively?
  2. OpenCoder Integration - Does OpenCoder use ContextScout for unfamiliar patterns?
  3. ContextScout Functionality - Does ContextScout discover context correctly?

Test Structure

evals/agents/
├── core/
│   ├── openagent/tests/contextscout-integration/
│   │   ├── 01-known-context-direct-load.yaml          # Should NOT use ContextScout
│   │   ├── 02-unknown-domain-discovery.yaml           # Should use ContextScout
│   │   ├── 03-accuracy-correct-files.yaml             # ContextScout finds right files
│   │   ├── 04-implicit-discovery.yaml                 # NEW: Proactive usage
│   │   ├── 05-multi-domain-comprehensive.yaml         # NEW: Multi-domain discovery
│   │   └── README.md
│   │
│   └── opencoder/tests/contextscout-integration/
│       ├── 01-implicit-pattern-discovery.yaml         # NEW: Pattern discovery
│       └── README.md
│
└── ContextScout/tests/
    ├── 01-code-standards-discovery.yaml               # NEW: Basic discovery
    ├── 02-domain-specific-discovery.yaml              # NEW: Domain-specific
    ├── 03-bad-request-handling.yaml                   # NEW: Error handling
    ├── 04-multi-domain-comprehensive.yaml             # NEW: Multi-domain
    ├── 05-tool-usage-validation.yaml                  # NEW: Read-only enforcement
    └── README.md

Test Categories

Category A: OpenAgent Integration (6 tests)

Location: evals/agents/core/openagent/tests/contextscout-integration/

Test Purpose Expected Behavior
01-known-context Validate direct loading for known tasks Should NOT use ContextScout
02-unknown-domain Validate discovery for unfamiliar topics Should use ContextScout
03-accuracy Validate ContextScout finds correct files Finds MVI.md correctly
04-implicit-discovery NEW: Proactive usage without instruction Uses ContextScout automatically
05-multi-domain NEW: Comprehensive multi-domain discovery Finds all relevant files

Read the full file on GitHub · 316 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 3d ago First seen · 316 lines · 0 tokens per session scan A b119bd9eb812

Subscribe to this mod's changes

CONTEXTSCOUT_INTEGRATION_TESTS is an agent published in the GitHub repository darrenhinde/OpenAgentsControl (4,815 stars, last pushed yesterday), licensed MIT. It costs nothing until one of its globs matches a file; then it loads 2,527 tokens. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.