Adding parallel workers lets a system finish more requests per second without making any single request faster, and one shared step that only one request can use at a time puts a hard ceiling on the whole path. This article separates latency from throughput with a small simulation, shows where the extra time goes when a 50 ms lock sits in the path, and explains how to find that lock in a real system.
Prerequisites
A request spends its time in two ways. It is either being worked on (CPU, a query running) or waiting for something busy (a free worker, a lock, a connection). Work time belongs to the request; waiting time depends on everything else happening at that moment.
Percentiles. The p50 (median) is the latency half of all requests beat; the p95 is the latency 95% beat, the slow end one request in twenty sees. A mean hides that tail.
Throughput ceilings from part 5. A resource with c parallel slots, each held S seconds per request, can finish at most c ÷ S requests per second. Part 5 used it to explain why a fast server can lose under load; here we apply it to a step with exactly one slot, a lock.
Definitions
Definition
Latency: the time one request takes from a chosen start point until it is done. Always say where the clock starts: at the client, at arrival in the queue, or (in our simulation) at worker pickup.
Definition
Throughput: requests finished per second. In our simulation, 1,000 requests divided by the batch time.
Definition
Worker (lane): a thread, process, container or server that handles one request at a time, in parallel with the others.
Definition
Serial step: a part of the request path that only one request can be inside at a time, such as a mutex, a row lock held by every write, or a single-threaded consumer. Its hold time per request sets a throughput ceiling of 1 ÷ hold time.
Definition
Lock contention: requests asking for a lock faster than it can be granted, so they wait. It shows up as lock wait time, not CPU.
Definition
Bottleneck: the resource with the highest demand relative to its capacity. It sets the maximum throughput of the whole path, however much capacity everything else has.
Definition
Little's law: in any stable system, the average number of items inside equals their throughput times the average time each spends inside: L = λ × W.
The same split is how working engineers explain it. Alex Xu, author of the System Design Interview books, put it this way on X: latency is "what users feel when they click a button", throughput is "how many packets flow through the pipe".
Post on X by @alexxubyte (Alex Xu, ByteByteGo). Screenshot; open the link for the original post. How it works: work, waiting and lanes
Write the latency of a request as a sum:
latency = own work + waiting = (time being processed) + (time in line for workers, locks, connections, disks) Adding workers changes only the second term. A request that needs 200 ms of CPU still needs 200 ms when nine others run beside it on other cores. More workers change how many 200 ms jobs happen per second, and how long a new request waits for a free worker. With nothing shared between workers:
throughput ≈ workers / work time (c / S) latency ≈ work time (once a worker has picked the request up) So 1 worker at 200 ms per request finishes 1 ÷ 0.2 s = 5 requests a second; 10 workers finish 10 ÷ 0.2 s = 50. Latency from pickup stays at 200 ms. Scaling out buys more output per second at the same speed per item.
One lane: 5 requests a second, each taking a median of 200 ms. Where the clock starts matters
Our simulation starts the latency clock at pickup. That isolates the work and any waiting after pickup (for the lock), and deliberately leaves out time spent in the backlog before a worker was free. In a real service that backlog wait is often most of what a user feels, and more workers do shrink it. Two teams can both be right about "latency" while measuring different intervals.
Adding a shared step
Now suppose every request, after its own 200 ms of work, must hold one shared lock for 50 ms. Think of one hot database row, a shared map under a mutex, or a counter every request increments. The lock is granted first come, first served. Two things change:
- The work per request grows from 200 ms to 250 ms, so with a single worker latency becomes 250 ms and throughput 1 ÷ 0.25 s = 4 req/s.
- A new ceiling appears: one holder at a time for 50 ms means at most 1 ÷ 0.05 s = 20 grants a second, whatever the number of workers.
The throughput of the whole path is whichever limit is lower:
X = min( workers / (work + hold), 1 / hold ) = min( c / 0.25 s, 20 req/s ) For 1, 2 and 5 workers the first term gives 4, 8 and 20 req/s; at 5 the limits meet. At 10 the lanes could do 40 req/s, the lock allows 20, and the surplus can only become waiting.
What the simulation measured
To check the arithmetic we wrote a discrete-event simulation: a virtual clock and a list of timed events (work finished, lock granted), jumping from one event to the next. No real threads, scheduler, garbage collector, network or disk, so machine speed does not affect the numbers (Node.js v22.13.1 on an Apple M2 Pro, seed 42). Each run is a backlog of 1,000 requests with 200 ms ± 10 ms of uniform work; the same 1,000 work times are reused in every run, so only the worker count changes. Latency is timed from pickup to done.
| Workers | Batch time | Throughput | p50 | p95 |
|---|---|---|---|---|
| 1 | 200.2 s | 5.0 req/s | 200 ms | 209 ms |
| 2 | 100.2 s | 10.0 req/s | 200 ms | 209 ms |
| 5 | 40.1 s | 24.9 req/s | 200 ms | 209 ms |
| 10 | 20.1 s | 49.8 req/s | 200 ms | 209 ms |
Run 1, independent lanes. Throughput rises in step with workers: 5, 10, 24.9, 49.8 req/s, within half a percent of c ÷ 0.2 s (the shortfall is lanes going idle at the end of the batch). Latency does not move: p50 200 ms and p95 209 ms at every count.
Ten lanes: 49.8 requests a second, and the median is still 200 ms. | Workers | Batch time | Throughput | p50 | p95 | Avg lock wait |
|---|---|---|---|---|---|
| 1 | 250.2 s | 4.0 req/s | 250 ms | 259 ms | 0 ms |
| 2 | 125.3 s | 8.0 req/s | 251 ms | 259 ms | 0 ms |
| 5 | 51.3 s | 19.5 req/s | 256 ms | 260 ms | 6 ms |
| 10 | 50.2 s | 19.9 req/s | 500 ms | 500 ms | 249 ms |
Run 2, one shared 50 ms lock. Each row is a different regime.
- 1 worker: 250 ms, no waiting. The only request in flight always finds the lock free: 200 ms of work plus 50 ms of holding the lock, average wait 0, throughput 1 ÷ 0.25 s = 4.0 req/s.
- 2 workers: almost no waiting. Throughput doubles to 8.0 req/s. The 251 ms p50 shows the two requests occasionally reaching the lock together; the average wait still rounds to 0.
- 5 workers: at the knee. Five lanes ask for the lock about 5 ÷ 0.25 s = 20 times a second, exactly its capacity. It is busy 19.5 × 0.05 = 97.5% of the time, waits appear (6 ms average) and throughput is 19.5 req/s.
- 10 workers: past the ceiling. 19.9 req/s, barely above 5 workers, and the median doubles to 500 ms. Five extra workers bought 0.4 req/s and about 249 ms of waiting per request.
The shared lock sits across every lane; latency is counted from pickup to done. Where the 500 ms comes from
The 10-worker latency splits exactly into its parts:
1 worker: 200 ms work + 0 ms lock wait + 50 ms lock hold = 250 ms 10 workers: 200 ms work + ~249 ms lock wait + 50 ms lock hold ≈ 500 ms The work time is the same in both rows, so every extra millisecond is lock wait.
The 10-worker p95 is also exactly 500 ms because a lock busy all the time grants 50 ms slots back to back. Ten lanes take turns, so each lane gets the lock once every 10 × 50 = 500 ms, and each request takes exactly one turn. Work jitter disappears, since a request that finishes early just waits longer, and the saturated lock sets the pace for every lane.
Why one serial step caps everything
The lock is a resource with one slot: c ÷ S = 1 ÷ 0.05 s = 20 req/s, and Run 2 converges on it. This is the bottleneck bound. Each request places a demand D on each resource (time per request ÷ parallel slots), and:
X ≤ 1 / D_max (the busiest resource sets the ceiling) X ≤ N / (D_1 + D_2 + ... + D_k) (with N requests in flight and no waiting at all) Run 2: D_workers = 0.25 s / c, D_lock = 0.05 s / 1 c = 10 → D_workers = 0.025 s, D_lock = 0.05 s → the lock is the bottleneck These bounds come from operational analysis, as in Quantitative System Performance (see sources). More workers lower D_workers but cannot touch D_lock, so the ceiling stays at 20 req/s. To raise it you have to hold the lock for less time, split it into independent locks, or remove it from the path.
Amdahl's law says the same thing about speedup
Amdahl's law gives the speedup of a job whose fraction s must run serially: speedup(N) = 1 ÷ (s + (1 − s) ÷ N), never more than 1 ÷ s. Here s = 50 ÷ 250 = 0.2, so at most 5 times one worker: 5 × 4 = 20 req/s, the same ceiling.
The curves differ before the ceiling. Amdahl assumes nothing else runs during the serial part, giving 1 ÷ (0.2 + 0.08) ≈ 3.6 times at 10 workers, about 14.3 req/s. Our other lanes keep working while one request holds the lock, so throughput reaches the ceiling much sooner (19.9 req/s). Treat Amdahl as the pessimistic curve and 1 ÷ D_max as the limit both agree on.
Often the serial step exists for correctness: a seat must not be sold twice, as in how an atomic update stops two fans buying one seat. Keep the lock; shrink what happens inside it.
The real-world caveat: the backlog our metric does not count
Run 2's 10-worker median of 500 ms looks worse than the 1-worker 250 ms, but for whoever is at the back of the line it is far better. The last of the 1,000 requests finished after 250.2 s with one worker and after 50.2 s with ten. Our metric starts at pickup, so it never sees the minutes the 1,000th request waited for the single worker. In a real system, more workers also shrink that pre-pickup queue, so end-to-end user wait can improve even when pickup-to-done p50 does not.
The same data shows where that stops. From 5 to 10 workers the batch time went from 51.3 s to 50.2 s, about 2%. Past the lock's ceiling, extra workers only move waiting from the backlog into the lock's line; the worker count decides where the wait is spent and which dashboard shows it.
Reading p50 and p95 honestly
Run 1's narrow gap (200 against 209 ms) is just the ±10 ms work jitter. Run 2 at 1 worker shifts both by the 50 ms hold (250 and 259 ms). At 10 workers the gap closes (500 and 500 ms) because the saturated lock imposes one rhythm. Three lessons for real dashboards:
- Identical printed percentiles do not prove identical distributions. Run 1's four rows really are the same distribution, but only because the code reuses the same 1,000 work times. On a dashboard you cannot know that. If 1% of requests take 2 s, the p95 does not move at all. Percentiles are a few rounded points on a curve.
- A narrow gap at a high value often means one saturated resource: everyone waits behind the same line. A wide gap points at something hitting only some requests, such as slow queries or garbage collection pauses.
- Do not average percentiles. The mean of ten servers' p95 values is not the fleet's p95. Merge histograms first, then compute percentiles.
The Google SRE monitoring chapter (see sources) agrees: collect latency as a distribution, and track failed requests separately so fast errors do not flatter it.
Little's law ties the two together
Latency and throughput are not independent knobs. For any stable system (Little, 1961):
L = λ × W L average number of requests inside the system λ throughput (arrival rate = completion rate when stable) W average time each request spends inside Apply it to the 10 workers. In both runs every lane is busy all the time, so about 10 requests are inside:
Run 1, 10 workers: L = 49.8 req/s × 0.2 s ≈ 10 (all ten lanes doing work) Run 2, 10 workers: L = 19.9 req/s × 0.5 s ≈ 10 (same ten lanes, half the throughput) With L fixed at 10, cutting λ from about 50 to about 20 must stretch W from 0.2 to 0.5 s. The law is also the fastest sanity check on a load test: throughput × latency should equal the concurrency you configured; if not, something is counting a different interval.
The law works on any box you draw. Three boxes for Run 2 at 10 workers:
doing its own work: 19.9 req/s × 0.200 s ≈ 3.98 requests holding the lock: 19.9 req/s × 0.050 s ≈ 1.00 request (lock busy 99.5% of the time) waiting for the lock: 19.9 req/s × 0.249 s ≈ 4.96 requests total ≈ 9.9 ≈ 10 workers Half of the lanes are parked at the lock, which is why a lock-bound service looks calm on CPU graphs: waiting threads use no CPU. The lock's utilisation (throughput × hold time) is 99.5%. Part 1 walks through the curve behind this in performance and scalability for a ticket rush: waiting grows slowly at moderate utilisation and very fast near 100%.
The same law turns requirements into concurrency: 300 requests a second (assumed) at 200 ms each keeps 300 × 0.2 = 60 in flight, so at least 60 workers before headroom. Getting the 300 is the subject of requirements, SLOs and capacity estimation.
Finding the toll booth in a real system
In production the serial step rarely has "LOCK" painted on it. The signature is the same: throughput flattens while latency rises, CPU stays moderate, and more instances do not help. The USE method (see sources) gives a checklist: for every resource check Utilisation, Saturation and Errors. It includes software resources such as mutex locks, where utilisation is the time held and saturation is the number of threads queued for it.
In PostgreSQL
- Who is waiting right now:
pg_stat_activityshows each session'swait_event_type; a steady set showingLockmeans lock contention. Thepg_locksview lists every lock, granted or not, andpg_blocking_pids(pid)names the sessions blocking a given one. - Waits over time:
log_lock_waits = onlogs any lock wait longer thandeadlock_timeout(1 s by default). - The usual hot spots: an UPDATE on one counter row from every request, or
SELECT ... FOR UPDATEon a shared parent row. The explicit locking documentation lists which lock modes conflict; a row lock taken by an UPDATE blocks other writers of that row until commit, so the hold time is the rest of the transaction, not just the statement.
In application code
- Go:
runtime.SetMutexProfileFractionand the block profile inpprofshow where goroutines wait on mutexes. - Java: thread dumps that repeatedly show many threads BLOCKED on the same monitor point straight at it.
- Linux: off-CPU analysis (time threads spend blocked) finds waits a CPU profiler never sees.
- Your own metrics: a histogram of time to acquire each lock or pool, recorded separately from hold time, tells you more about contention than any other lock metric.
Confirm it with a step test
Raise concurrency in steps (1, 2, 5, 10, 20) and record throughput, p50, p95 and wait time at each, like the tables above. When throughput stops rising, compute the suspect's ceiling (slots ÷ hold time). If it matches the plateau and that resource's wait time is what grew, you have found the bottleneck. Fix it and rerun: the plateau should move by the predicted amount.
This is the "measure it" step of the design loop used in designing Instagram's feed in five steps: predict a number, measure it, and explain the gap.
Worked example
The video's picture: lanes are workers, cars are requests. Widening the road from 1 lane to 10 moves ten times as many cars, but each still drives the same 200 ms stretch. Then one toll booth taking 50 ms per car goes across all ten lanes. Inputs are simulation configuration; results come from its output.
With the booth in place, ten lanes give 19.9 requests a second at a 500 ms median. CONFIG 1,000 requests, work 200 ms ± 10 ms, lock 50 ms (run 2), seed 42 latency = pickup to done LANES c / S 1 / 0.2 s = 5 req/s measured 5.0 10 / 0.2 s = 50 req/s measured 49.8 latency 200 ms of work measured p50 200 ms at every count BOOTH ceiling 1 / 0.05 s = 20 req/s 1 worker 1 / 0.25 s = 4 req/s measured 4.0, p50 250 ms 10 workers min(40, 20) = 20 req/s measured 19.9 LITTLE W = L / X 10 / 19.9 req/s ≈ 0.50 s measured p50 500 ms split 200 work + 249 wait + 50 hold ≈ 500 ms BACKLOG last of 1,000 done: 1 worker 250.2 s, 5 workers 51.3 s, 10 workers 50.2 s The decision follows from the ceiling. Up to about 5 workers, adding lanes is the cheap win. Past 5, each new worker costs memory and connections, adds no throughput and raises latency. More throughput must come from the booth: a 25 ms hold (assumed) gives a 40 req/s ceiling; splitting the lock into 4 independent ones (assumed, for partitionable data) lifts the lock's limit to 80.
Code
The simulation behind every measured number, trimmed for reading. Same model, seed and event ordering as our lab script, and the same numbers. Save it as latency_lab.mjs and run node latency_lab.mjs (Node.js 18 or newer, no packages, no network).
// latency_lab.mjs: more lanes, same work. Discrete-event simulation, virtual clock. // 1,000 requests, each 200 ms +/- 10 ms of its own work. Run 2 adds one shared // 50 ms lock (first come, first served) after the work. Latency = pickup to done. // Node.js 18+, no packages, no network. Same seed, same numbers, any machine. const SEED = 42, REQUESTS = 1000, WORK_MS = 200, JITTER_MS = 10, LOCK_MS = 50; const WORKER_COUNTS = [1, 2, 5, 10]; function mulberry32(seed) { // small seeded random number generator let a = seed >>> 0; return () => { a = (a + 0x6d2b79f5) >>> 0; let t = a; t = Math.imul(t ^ (t >>> 15), t | 1); t ^= t + Math.imul(t ^ (t >>> 7), t | 61); return ((t ^ (t >>> 14)) >>> 0) / 4294967296; }; } // The same 1,000 work times for every run: only the number of lanes changes. const rand = mulberry32(SEED); const work = Array.from({ length: REQUESTS }, () => WORK_MS - JITTER_MS + rand() * 2 * JITTER_MS); function simulate(workers, useLock) { const events = []; // "work done" events, sorted by (time, sequence) let seq = 0, next = 0, lockFreeAt = 0, lockWait = 0, last = 0; const latencies = []; const push = (t, start) => { const e = { t, s: seq++, start }; let lo = 0, hi = events.length; while (lo < hi) { const m = (lo + hi) >> 1; if (events[m].t < t || (events[m].t === t && events[m].s < e.s)) lo = m + 1; else hi = m; } events.splice(lo, 0, e); }; const pickUp = (t) => { if (next < REQUESTS) push(t + work[next++], t); }; for (let w = 0; w < workers; w++) pickUp(0); // every lane takes one request while (events.length) { const { t, start } = events.shift(); // this request's own work just ended let done = t; if (useLock) { // the one shared step const granted = Math.max(t, lockFreeAt); // wait if someone holds the lock lockWait += granted - t; lockFreeAt = granted + LOCK_MS; done = lockFreeAt; } latencies.push(done - start); // counted from pickup, not from arrival last = Math.max(last, done); pickUp(done); // lane is free, take the next request } latencies.sort((a, b) => a - b); const pct = (p) => Math.round(latencies[Math.ceil(p * REQUESTS) - 1]); return { workers, rps: (REQUESTS / (last / 1000)).toFixed(1), p50: pct(0.5), p95: pct(0.95), wait: Math.round(lockWait / REQUESTS) }; } for (const useLock of [false, true]) { console.log(useLock ? `\nshared ${LOCK_MS} ms lock` : "independent lanes"); console.log("workers req/s p50 ms p95 ms avg lock wait ms"); for (const w of WORKER_COUNTS) { const r = simulate(w, useLock); console.log(`${String(w).padStart(7)} ${r.rps.padStart(7)} ${String(r.p50).padStart(8)} ` + `${String(r.p95).padStart(8)} ${String(r.wait).padStart(18)}`); } } Real output:
independent lanes workers req/s p50 ms p95 ms avg lock wait ms 1 5.0 200 209 0 2 10.0 200 209 0 5 24.9 200 209 0 10 49.8 200 209 0 shared 50 ms lock workers req/s p50 ms p95 ms avg lock wait ms 1 4.0 250 259 0 2 8.0 251 259 0 5 19.5 256 260 6 10 19.9 500 500 249 The prediction side in plain Python 3, no simulation:
# predict.py: bottleneck law and Little's law, before you run anything. WORK_S, LOCK_S = 0.200, 0.050 # work per request, lock hold per request for workers in (1, 2, 5, 10): lanes = workers / WORK_S # Run 1 ceiling: c / S with_lock = min(workers / (WORK_S + LOCK_S), 1 / LOCK_S) # Run 2: lanes or lock, whichever is lower latency_ms = workers / with_lock * 1000 # Little's law: W = L / X print(f"{workers:>2} workers no lock {lanes:5.1f} req/s " f"lock {with_lock:5.1f} req/s, latency about {latency_ms:4.0f} ms") 1 workers no lock 5.0 req/s lock 4.0 req/s, latency about 250 ms 2 workers no lock 10.0 req/s lock 8.0 req/s, latency about 250 ms 5 workers no lock 25.0 req/s lock 20.0 req/s, latency about 250 ms 10 workers no lock 50.0 req/s lock 20.0 req/s, latency about 500 ms The formula misses only at the knee: at 5 workers it predicts 20 req/s and 250 ms against a measured 19.5 req/s and 256 ms, because random work times make requests collide at the lock even when average demand equals capacity. Real systems lose more there, so plan a serial step well below 100% utilisation.
What to observe when you change the simulation:
- Set
LOCK_MSto 25. The ceiling becomes 40 req/s; 10 workers reach 39.7 req/s and p50 falls to 250 ms. Halving the critical section doubles the ceiling, which no number of workers could do. - Set
WORKER_COUNTSto[10, 20, 50]. With the lock, throughput stays at 19.9 req/s and p50 grows as workers × 50 ms: 1,000 ms at 20 workers, 2,500 ms at 50.
Failure modes and edge cases
Autoscaling on latency into a lock. An autoscaler that adds instances when p95 rises keeps adding them, because more instances make lock waits longer. Detect: flat throughput while instance count climbs. Fix: scale on the bottleneck's saturation and cap instances.
Lock held across slow work. A transaction that takes a row lock and then calls an external API turns the hold time into that call's time. A 50 ms hold becomes 500 ms during a partner's slow minute, and the ceiling drops from 20 to 2 req/s. Detect with traces that show calls inside transactions; fix by doing slow work before taking the lock or after releasing it.
Measuring the wrong interval. Server-side latency usually starts after the request leaves the accept or thread pool queue, so a saturated service can report a stable p50 while clients time out. Compare client-side and server-side latency, and record queue time as its own metric.
Retries against a saturated serial step. Client timeouts shorter than the lock wait cause retries, which add demand to a resource already at 100%. Fix with backoff, jitter and early rejection, as covered for overload in the ticket rush article.
Splitting a lock that guards an invariant. Sharding one counter removes contention, but "never sell more than 5,000 seats" then needs per-shard quotas or a final check, or the faster version can oversell.
Trade-offs
| Option | Good for | Costs you | Use when |
|---|---|---|---|
| Add workers or instances | Throughput on independent work; drains backlogs faster | Memory, connections, money; no help past a serial ceiling | Utilisation of the workers is the highest demand on the path |
| Shorten the critical section | Raises the 1 ÷ hold ceiling directly | Code changes; may need to restructure transactions | Lock wait dominates latency and work can move outside the lock |
| Split the lock (sharding, striping) | Ceiling grows with the number of independent locks | Harder invariants across shards; uneven keys create new hot spots | The protected data partitions naturally (per user, per seat block) |
| Batch inside the lock | One acquisition serves many requests | Added latency while a batch fills; larger failure blast radius | Hold time is mostly fixed overhead (commit, fsync) rather than per-item work |
| Remove the lock (atomic operations, append-only log, single writer queue) | No waiting on the hot path | Different consistency model; work moves to a background consumer | The result can be applied asynchronously or with a single atomic instruction |
| Limit concurrency in front of the bottleneck | Bounded waits, predictable latency for admitted requests | Some requests rejected or delayed at the edge | Arrivals can exceed the serial ceiling |
Common mistakes
Mistake: Using "fast" for both latency and throughput. Instead: Say which one: "200 ms per request" or "50 requests per second", and at what concurrency.
Mistake: Expecting more servers to make each request faster. Instead: More servers add throughput and cut queue time; to cut work time, make the work itself smaller.
Mistake: Adding workers when throughput has flattened. Instead: Find the resource whose ceiling matches the plateau (slots ÷ hold time) and work on that.
Mistake: Reading low CPU as spare capacity. Instead: Check saturation of locks, pools and queues; waiting threads use no CPU.
Mistake: Quoting latency without saying where the clock starts. Instead: State the interval (client to client, arrival to done, pickup to done) and measure queue time separately.
Mistake: Treating equal p50 and p95 as proof that two systems behave the same. Instead: Compare full histograms, including p99 and the maximum.
Explaining it in 60 seconds
Explaining a flat throughput graph to your team or reviewing your friend's scaling plan, six steps keep it short. (They also work as a system design interview answer.)
- Clarify: which latency interval, which percentile, what target throughput.
- Assumptions: work time per request, and any one-at-a-time step.
- Structure: throughput = min(workers ÷ work time, 1 ÷ serial hold time); latency = work + waiting.
- Deep dive: Little's law to check numbers; wait metrics to find the serial step.
- Trade-offs: more workers against a shorter, split or removed critical section.
- Wrap up: the change and the step test that proves the plateau moved.
A 60-second answer
Latency is how long one request takes; throughput is how many finish per second. In our simulation each request needs 200 ms of work, so going from 1 to 10 workers raised throughput from 5 to 49.8 requests a second while the median stayed at 200 ms: more lanes, same speed per car. Then I add one shared 50 ms lock. A lock serves one request at a time, so it caps throughput at 1 ÷ 50 ms = 20 a second. Ten workers reached only 19.9, and the median rose to 500 ms: 200 of work, about 249 waiting, 50 holding. Little's law agrees: 10 in flight ÷ 19.9 per second is about half a second. Up to the ceiling, more workers still drain a backlog faster; past ...

