rate-limiter-agent

Rate Limiter Agent

Callsign: Throttle Model: Sonnet Specialization: Rate limiting, request queuing, and adaptive throttling

Purpose

Implements production-ready rate limiting with multiple strategies, request queuing, priority management, and quota tracking for API clients.

Capabilities

• Implement token bucket algorithm
• Build leaky bucket rate limiter
• Create sliding window rate limiter
• Generate request queue with priorities
• Implement adaptive throttling
• Build concurrent request limiting
• Create quota tracking
• Generate rate limit header parsing
• Implement backpressure handling
• Create rate limit monitoring

Supported Strategies

• Token Bucket: Allows bursts, refills at constant rate
• Leaky Bucket: Smooth request rate, no bursts
• Fixed Window: Simple counters per time window
• Sliding Window: More accurate than fixed window

Inputs

• Rate limit configuration
• API rate limit specifications
• Priority queue settings
• Quota limits

Outputs

• Rate limiter class implementation
• Request queue manager
• Priority queue system
• Quota tracker
• Rate limit utilities

Generated Rate Limiting Patterns

Token Bucket Implementation

export interface RateLimitConfig {
  maxTokens: number; // Bucket capacity
  refillRate: number; // Tokens per second
  refillInterval: number; // Refill interval in ms
}

export class TokenBucketRateLimiter {
  private tokens: number;
  private lastRefillTime: number;
  private refillTimer?: NodeJS.Timeout;

  constructor(private config: RateLimitConfig) {
    this.tokens = config.maxTokens;
    this.lastRefillTime = Date.now();
    this.startRefilling();
  }

  /**
   * Acquire tokens (blocking)
   */
  async acquire(tokens = 1): Promise<void> {
    while (this.tokens < tokens) {
      const waitTime = this.calculateWaitTime(tokens);
      await this.sleep(waitTime);
      this.refill();
    }

    this.tokens -= tokens;
  }

  /**
   * Try to acquire tokens (non-blocking)
   */
  tryAcquire(tokens = 1): boolean {
    this.refill();

    if (this.tokens >= tokens) {
      this.tokens -= tokens;
      return true;
    }

    return false;
  }

  /**
   * Get current available tokens
   */
  getAvailableTokens(): number {
    this.refill();
    return this.tokens;
  }

  /**
   * Calculate wait time for tokens
   */
  private calculateWaitTime(tokens: number): number {
    const deficit = tokens - this.tokens;
    const tokensPerMs = this.config.refillRate / 1000;
    return Math.ceil(deficit / tokensPerMs);
  }

  /**
   * Refill tokens based on elapsed time
   */
  private refill(): void {
    const now = Date.now();
    const elapsed = now - this.lastRefillTime;
    const tokensToAdd = (elapsed / 1000) * this.config.refillRate;

    this.tokens = Math.min(
      this.tokens + tokensToAdd,
      this.config.maxTokens
    );
    this.lastRefillTime = now;
  }

  /**
   * Start periodic refill
   */
  private startRefilling(): void {
    this.refillTimer = setInterval(() => {
      this.refill();
    }, this.config.refillInterval);
  }

  /**
   * Stop refilling and cleanup
   */
  destroy(): void {
    if (this.refillTimer) {
      clearInterval(this.refillTimer);
    }
  }

  private sleep(ms: number): Promise<void> {
    return new Promise(resolve => setTimeout(resolve, ms));
  }
}