105-context-trim

context-trim.mdc

• Purpose: To maximize effective context utilization by intelligently compressing input data while preserving semantic integrity. This rule acts as a pre-processor that ensures optimal signal-to-noise ratio, enabling deeper reasoning within computational constraints. Proven Performance: MInference achieves 10x speedup with 90-95% semantic preservation. LLMLingua demonstrates 20:1 compression ratios while maintaining task accuracy.

• Requirements:

  • Compression Strategies:
    • keep_edges: Preserve first 20% and last 20%, summarize middle
    • semantic_skeleton: Maintain structure markers (headings, key transitions)
    • perplexity_prune: Remove low-information tokens based on predictability
    • hierarchical_summary: Recursive summarization preserving relationships
    • task_aware: Adapt strategy based on task type (code vs. narrative vs. data)
  • Token Budgets:
    • Simple tasks: 4K tokens maximum
    • Moderate tasks: 16K tokens target
    • Complex tasks: 64K tokens with structured compression
    • Never exceed 128K without explicit justification
  • Preservation Priorities:
    1. User query and direct context
    2. Structural markers and transitions
    3. Unique/rare information (high perplexity)
    4. Causal relationships and dependencies
    5. Concrete examples and edge cases
  • Compression Techniques:
    • Replace verbose descriptions with concise labels
    • Extract and index repeated patterns
    • Convert narrative to structured data where possible
    • Use reference pointers for redundant content
    • Maintain running summary of trimmed content

• Validation:

  • Check: Is token count within specified budget?
  • Check: Are document structure markers preserved?
  • Check: Can key information be reconstructed from compressed version?
  • Check: Is compression strategy appropriate for content type?
  • Check: Is compression ratio logged for optimization?

• Examples:

  • Scenario: Large log file analysis (1M tokens)

    COMPRESSION PLAN:
    Input: Server logs (1M tokens, 50K lines)
    Strategy: hierarchical_summary + pattern_extraction
    
    Step 1: Pattern Detection
    - Identified: 5 error patterns (80% of content)
    - Extracted: Pattern templates + occurrence counts
    
    Step 2: Temporal Compression  
    - Kept: First/last 100 lines verbatim
    - Summarized: Hourly aggregates for middle section
    
    Step 3: Anomaly Preservation
    - Preserved: 47 unique errors (full context)
    - Indexed: Location pointers for drill-down
    
    Result: 15K tokens (93% reduction)
    Retained: All errors, patterns, temporal flow
    

  • Scenario: Multi-document research (300K tokens)

    COMPRESSION EXECUTION:
    Documents: 12 research papers
    
    Semantic Skeleton:
    - Title, Abstract, Conclusions: KEEP FULL
    - Methods: EXTRACT key algorithms only  
    - Results: CONVERT tables to key findings
    - Discussion: SUMMARIZE to main arguments
    - References: INDEX by relevance score
    
    Cross-Document:
    - Deduplicate common background (40% reduction)
    - Create citation graph (relationships preserved)
    - Build unified terminology map
    
    Output: 45K tokens with full semantic coverage
    

• Advanced Patterns:

  • Attention-Based Trimming: Use A-shape pattern for documents, Block-Sparse for code
  • Sliding Window: Maintain detailed context for recent tokens, aggressive compression for older
  • Semantic Chunking: Compress at natural boundaries (paragraphs, functions, sections)
  • Lossless References: Store trimmed content with retrieval pointers
  • Progressive Disclosure: Start compressed, expand on demand

• Implementation Architecture:

  Raw Input → Tokenizer → Strategy Selector → Compressor → Validator → Output
                 ↓              ↓                ↓            ↓          ↓
           [Token Count]  [Content Type]  [Techniques]  [Fidelity]  [Context]