file-uploads

Handle file uploads and downloads safely

An upload is untrusted input that is large, binary, and often stored for years. Limit it before you read it, verify what it really is, write it under a name you chose, atomically, in a place that cannot execute, and serve it back with headers that stop the browser guessing. Source: OWASP File Upload Cheat Sheet; OWASP ASVS 5 V5 (file handling).

Steps

1. Use the repo's storage and upload path. Object storage with presigned URLs, a local store behind an abstraction, or existing multipart middleware. Call search_lore for bucket layout, key naming and size limits. Do not add a second storage mechanism.
2. Authorise and limit before reading. Check the caller may upload here (the security-authz skill) and that the form is CSRF-protected if cookie-authenticated. Enforce a maximum body size at the edge (proxy and framework), a maximum part count, and a request timeout; reject with 413 without buffering. Stream to storage; never hold a whole file in memory. For presigned uploads, sign the exact key, Content-Type and size limit (a content-length-range POST-policy condition, or a signed Content-Length for PUT), with a short expiry.
3. Validate content, not labels. Allow-list extensions (after normalising case and rejecting double extensions like x.jpg.php, NUL bytes and :), check the declared Content-Type against the allow-list, then detect the real type from the bytes (magic numbers) and reject any mismatch. None of these alone is sufficient.
4. Defuse the dangerous formats. Images: cap pixel dimensions before decoding (decompression bombs), then decode and re-encode to strip metadata and payloads. Archives: cap total uncompressed size and entry count, and reject entries whose resolved path escapes the target directory (zip slip). SVG, HTML and XML are active content: refuse them, sanitise them, or serve them only as attachments; parse XML with external entities disabled (XXE). CSV/XLSX imports: cap rows and cell size, treat every cell as untrusted text, and on export prefix cells starting with =, +, -, @, tab or CR with ' (formula injection).
5. Generate the storage key; never trust the filename. Key = random id (UUID) plus an extension derived from the detected type. Keep the original name only as sanitised display metadata (length-capped, no path separators, no leading dot).
6. Write atomically and collision-free. Concurrent uploads are normal. Write to a unique temporary key (random suffix, never a timestamp or the user's name), then rename/commit to the final key; object storage PUT is already atomic per key. Insert the metadata row in the same transaction that marks the file available, or use a pending → available status so a half-written file is never served. Replacing a file (a new avatar) writes a new key and swaps the pointer with a conditional update (WHERE version = ?); it never overwrites bytes in place. Orphaned temp objects are swept by a job (the scheduled-jobs skill).
7. Store where nothing executes. Object storage or a directory outside the web root, no execute permission, no per-directory config overrides; private by default and access-checked on every read.
8. Serve safely. Presigned GET with a short expiry, or an authenticated route that sets Content-Disposition: attachment for anything not meant to render inline, the stored detected Content-Type, X-Content-Type-Options: nosniff, and ideally a separate cookieless origin with a restrictive CSP (default-src 'none') so a malicious file cannot run in the app's origin.
9. Scan and quarantine where the repo requires it (malware scan, CDR, moderation) asynchronously through a job (the background-jobs skill); the file stays unavailable until cleared.