scale-cliff

The silent hard limit and scale cliff

The code is correct for the median account and fatal for the large one. A fixed timeout, a cap on bound parameters, an identifier length limit, a row maximum, a counter that overflows, or an operation whose cost grows with a dimension nobody sized for. It does not degrade. It works, and then at some size it does not.

Symptom

The reporter says it works for them and not for a colleague, or that it worked until the account grew. A request times out only for the largest tenant. An export comes back short. A list stops at a suspiciously round count. A name is accepted and then appears truncated. A job stays in progress forever for one customer and finishes everywhere else.

The tell to look for later is that failure tracks a size rather than a time.

What Crumbtrail can see

Every request carries its timing, so a session shows exactly where duration climbed and where a request stopped completing. Named detector signals aimed at this shape include slow_request, latency_outlier, pending_request, result_row_loss, response_exceeded_requested_limit, response_count_mismatch, pagination_first_page_offset, accepted_text_was_truncated, job_did_not_complete, job_drain_left_work_deferred, n_plus_one_query and request_reconnect_storm.

Where the database adapter is installed, the session carries the rows that changed and the query activity that changed them, which is how an operation whose cost grows per row becomes visible. Where the backend integration is installed, backend spans carry the server side timing for the same request.

It does not see server memory, processor load, connection pool occupancy, garbage collection, or a database query plan. It cannot tell you the value of the limit. It can tell you the input size at which the request stopped succeeding, which is usually enough to name the dimension and let the code reveal the constant.

One capture bound matters enough to state before you use it. A response body larger than the configured maximum is summarised rather than stored, and the summary records that reason. A short looking body is therefore not proof that the application truncated anything. Read the body summary reason first, and only call it product truncation when the reason is not a capture size bound.