QPS & Latency Calculator

Frequently Asked Questions

How do QPS and latency determine concurrency?

By Little's Law, average concurrency = arrival rate × average latency. At 200 QPS with 50 ms (0.05 s) average latency, you need about 200 × 0.05 = 10 requests in flight.

How many worker threads or connections do I need?

Provision at least the Little's-Law concurrency, then add headroom for variance. For 10 average in-flight requests, sizing a pool of ~15–20 absorbs latency spikes without queueing.

What QPS can a fixed pool sustain?

Rearranging Little's Law, max QPS = concurrency ÷ latency. A pool of 50 connections each handling 100 ms requests caps out near 50 ÷ 0.1 = 500 QPS.

Why does latency rise as I approach max QPS?

As utilization nears 100%, queueing delay grows nonlinearly (per queuing theory, roughly 1 ÷ (1 − utilization)). Running below ~70–80% utilization keeps tail latency stable.

Provided by AllCalculators.io
Free online calculators for everyday. No registration required.

Estimates for informational purposes only.

Important Disclaimer: Estimates for informational purposes only.

This calculator provides estimates for informational purposes only. Results are based on assumptions and may not reflect actual outcomes. Consult qualified professionals in relevant fields before making important decisions based on these results.

JavaScript is required to use the interactive calculator above. The questions and answers below remain readable without JavaScript.

How This Calculator Works

Little's Law is the single most useful equation in capacity planning. It states that the average number of requests being processed at any instant - the concurrency, often called L - equals the arrival rate (QPS, queries per second) multiplied by the average time each request spends in the system (latency). In symbols: L = QPS × latency. The beauty is that it holds for any stable system regardless of internal details: no assumptions about distributions or scheduling. This calculator lets you solve for whichever of the three you do not know. Give it QPS and average latency and it returns the concurrency you must support - and, applying a headroom factor, the number of worker threads or connections to provision. Give it concurrency and latency to find the QPS a fixed thread pool can sustain, or concurrency and QPS to back out the latency that implies. It is the back-of-the-envelope tool for sizing thread pools, connection pools, and worker fleets.

Formula / Method

Little's Law: L = λ × W, where λ is the arrival rate (QPS) and W is the average time in system (latency, converted from milliseconds to seconds). Rearranged: QPS = L ÷ W and latency W = L ÷ λ. To size a worker pool we divide the required concurrency by a target utilization (headroom): threads = ceil(L ÷ utilization). Running a pool at 100% utilization leaves no slack for variance, queueing, or latency spikes, so a common rule is to target 60–80% utilization - the calculator defaults to 70%. Latency must be the average end-to-end residence time (including any queueing), not the median or a percentile, for the law to apply exactly.

Worked Example

A service handles 500 QPS with an average latency of 40 ms (0.04 s). Concurrency L = 500 × 0.04 = 20 requests in flight on average. If you provision one thread per in-flight request you need 20, but running flat-out is risky - at 70% target utilization you provision ceil(20 ÷ 0.70) = ceil(28.57) = 29 threads or connections. Flip the question: a fixed pool of 20 threads at 40 ms latency sustains 20 ÷ 0.04 = 500 QPS - consistent, as it must be. And if you observe 20 concurrent requests at 500 QPS, the implied average latency is (20 ÷ 500) × 1000 = 40 ms. The three views are just rearrangements of the same identity.

Common Mistakes

Using a latency percentile (p95/p99) instead of the mean - Little's Law requires the average residence time.

Sizing a pool for the average and forgetting bursts - always add headroom; a pool at 100% utilization queues and latency explodes.

Forgetting to convert milliseconds to seconds - a 1,000× error in concurrency.

Applying it to an unstable system - if arrivals exceed capacity the queue grows without bound and the law no longer gives a finite L.

Confusing throughput with concurrency - 500 QPS at 40 ms is only 20 simultaneous requests, not 500 threads.

Tips & FAQ

What utilization should I target? 60–80% for steady services; lower if traffic is spiky, since queueing delay rises sharply as you approach 100%. Does this account for queueing delay? Only if your latency figure already includes time spent waiting in queue - measure end-to-end. Threads or connections? The same number applies to either - any resource that one in-flight request occupies for its full latency. How does this relate to the M/M/1 queue? Little's Law is more general - it needs no distribution assumptions, while queueing models add them to predict the latency itself. What if latency depends on load? Then iterate: higher concurrency often raises latency, so re-measure and recompute until it stabilizes. Async or event-loop servers? Concurrency maps to outstanding requests, not OS threads - size your connection pool and downstream limits to L instead. Use this as a first-cut sizing estimate, then validate with a load test under realistic traffic.

Frequently Asked Questions

Related Developer Tools Calculators