How fast will a linear search through all backends work at high RPS (for example, 30,000 RPS and 10 backends)?

A linear search through all backends at high RPS (for example, 30,000 requests per second and 10 backends) will be very inefficient and is likely to become a bottleneck in the system. In such an approach, each request is processed sequentially, checking each backend one by one, which increases latency and reduces throughput.

For high loads, parallel or asynchronous calls, load balancing, caching, and other optimizations are usually used. Linear search may be acceptable for small loads, but at 30,000 RPS, it will lead to significant delays and a risk of backend overload.

Example: if each backend responds in 1 ms, then 10 backends sequentially take up to 10 ms per request, which at 30,000 RPS already totals 300,000 ms of total time, which is impossible without parallelism.