aws-v4-signing-custom-headers-gcs

AWS v4 Signing with Custom Metadata Headers for GCS

Problem

When adding custom metadata headers (x-amz-meta-*) to GCS S3-compatible API requests, the requests fail with signature mismatch errors. The headers must be properly included in the AWS v4 signature calculation.

Context / Trigger Conditions

• Using GCS S3-compatible XML API (not the native JSON API)
• Adding x-amz-meta-* headers for object metadata
• Error: "SignatureDoesNotMatch" or 403 Forbidden after adding headers
• Headers work with unsigned requests but fail with AWS v4 signing
• HMAC credentials configured for GCS interoperability

Solution

Custom headers must be included in both the canonical headers and the signed headers list, sorted alphabetically:

1. Canonical Headers (alphabetically sorted)

let canonical_headers = format!(
    "host:{}\nx-amz-content-sha256:{}\nx-amz-date:{}\nx-amz-meta-owner:{}\n",
    host, payload_hash, amz_date, owner  // Note: alphabetical order!
);

2. Signed Headers List

let signed_headers = "host;x-amz-content-sha256;x-amz-date;x-amz-meta-owner";

3. Complete Example (Rust)

fn sign_request_with_owner(
    req: &mut Request,
    config: &GCSConfig,
    payload_hash: Option<String>,
    owner: &str
) -> Result<()> {
    // Set headers first
    req.set_header("x-amz-date", &amz_date);
    req.set_header("x-amz-content-sha256", &payload_hash);
    req.set_header("x-amz-meta-owner", owner);  // Custom metadata

    // Canonical headers - MUST be alphabetically sorted
    let signed_headers = "host;x-amz-content-sha256;x-amz-date;x-amz-meta-owner";
    let canonical_headers = format!(
        "host:{}\nx-amz-content-sha256:{}\nx-amz-date:{}\nx-amz-meta-owner:{}\n",
        host, payload_hash, amz_date, owner
    );

    // Rest of AWS v4 signing...
    let canonical_request = format!(
        "{}\n{}\n{}\n{}\n{}\n{}",
        method, uri, query, canonical_headers, signed_headers, payload_hash
    );

    // Sign and add Authorization header...
}