system-design-generator

System Design Generator

Create comprehensive system architecture plans from requirements.

System Design Document Template

# System Design: [Feature/Product Name]

## Overview

Brief description of what we're building and why.

## Requirements

### Functional

- User can upload videos (max 1GB)
- System processes video within 5 minutes
- User receives notification when complete

### Non-Functional

- Handle 1000 uploads/day
- 99.9% uptime
- Process videos in <5 minutes (p95)
- Cost: <$0.50 per video

## High-Level Architecture

┌─────────┐ ┌──────────┐ ┌─────────────┐ │ Client │─────▶│ API │─────▶│ Upload │ │ │ │ Gateway │ │ Service │ └─────────┘ └──────────┘ └─────────────┘ │ ▼ ┌─────────────┐ │ Storage │ │ (S3) │ └─────────────┘ │ ▼ ┌─────────────┐ │ Processing │◀─┐ │ Queue │ │ └─────────────┘ │ │ │ ▼ │ ┌─────────────┐ │ │ Processor │─┘ │ Workers │ └─────────────┘ │ ▼ ┌─────────────┐ │Notification │ │ Service │ └─────────────┘

## Components

### 1. API Gateway
**Responsibilities:**
- Authentication
- Rate limiting
- Request routing

**Technology:** Kong/AWS API Gateway
**Scaling:** Auto-scale based on requests/sec

### 2. Upload Service
**Responsibilities:**
- Generate pre-signed S3 URLs
- Validate file metadata
- Enqueue processing jobs

**API:**

POST /uploads Request: { filename, size, content_type } Response: { upload_url, upload_id }

**Technology:** Node.js + Express
**Scaling:** Horizontal (stateless)

### 3. Storage (S3)
**Responsibilities:**
- Store raw videos
- Store processed outputs
- Serve content via CDN

**Structure:**

/uploads/{user_id}/{upload_id}/original.mp4 /processed/{user_id}/{upload_id}/output.mp4

### 4. Processing Queue
**Responsibilities:**
- Buffer processing jobs
- Ensure at-least-once delivery
- DLQ for failed jobs

**Technology:** AWS SQS
**Configuration:**
- Visibility timeout: 15 minutes
- DLQ after 3 retries

### 5. Processor Workers
**Responsibilities:**
- Transcode videos
- Generate thumbnails
- Update database

**Technology:** Python + FFmpeg
**Scaling:** Auto-scale on queue depth

## Data Flow

### Upload Flow
1. Client requests upload URL from Upload Service
2. Upload Service generates pre-signed S3 URL
3. Client uploads directly to S3
4. Client notifies Upload Service of completion
5. Upload Service enqueues processing job
6. Returns upload_id to client

### Processing Flow
1. Worker polls queue for jobs
2. Downloads video from S3
3. Processes video (transcode, thumbnail)
4. Uploads results to S3
5. Updates database status
6. Sends notification
7. Deletes message from queue

## Data Model

```typescript
interface Upload {
  id: string;
  user_id: string;
  filename: string;
  size: number;
  status: 'pending' | 'processing' | 'complete' | 'failed';
  original_url: string;
  processed_url?: string;
  created_at: Date;
  processed_at?: Date;
}