State of the art in the optimal number of servers

The question of how many servers are best has a long history, but all the work is limited to FCFS (single priority class). Further, the question has not been fully resolved even for the case of FCFS. Note that the study of this question has been limited to FCFS primarily because an analysis of multiple priority systems involves multidimensional Markov chains, and exact (or nearly exact) analysis is known only for exponential job size distributions (see Section 3.3). In describing the results below, we will assume that the performance metric of interest is mean response time rather than mean delay (the time a job waits in the queue before receiving service), since in general an infinite number of servers minimizes mean delay [34,37] (also, Scheller-Wolf [172] shows that the tail of the stationary delay distribution is significantly reduced by increasing the number of servers under highly variable service demand distributions).

As early as 1958, Morse observed that for an M/M/$k$/FCFS system, the optimal number of servers is one ($k=1$) [131]. In 1970, Stidham formalized and generalized the observation by Morse[188]; Stidham showed that a single server is optimal in a G/Er/$k$/FCFS system, namely when the service demand has an Erlang distribution, including an exponential distribution, or is deterministic, for a general arrival process. Stidham's result is extended by Brumelle to the system where the service demand is at most as variable as the exponential random variable; i.e., a single server is optimal in a G/G/$k$/FCFS system where the service demand distribution has the squared coefficient variation at most one ($C^2\leq 1$) [31]. For general service demand distributions, only results for limiting cases are known. Specifically, the work by Reiman and Simon [161] can be used to show that a single server ($k=1$) is optimal in the light traffic limit (when servers are almost always idle) [120], and the work by Iglehart and Whitt [79] implies that a multiserver system (GI/GI/$k$/FCFS) and its single server counterpart (GI/GI/1/FCFS) coincides in the heavy traffic limit (when servers are almost always busy).

All the above work proves that a single server ($k=1$) is the best for special cases. However, Stidham also discusses that the optimality of a single server does not appear to hold for highly variable job size distributions [188]. Very recently, Molinero-Fernandez et. al. [130] consider the question of how many servers are best in an M/HT/$k$ single priority system, where HT denotes a heavy-tailed service distribution. To answer this question, they approximate a heavy-tailed distribution with a bimodal distribution, and observe that in general multiple servers are better than a single server.

In summary, although the question of ``how many servers are best?'' has been addressed in a number of papers in the context of FCFS, results are limited to special cases such as light and heavy traffic limits and low variable and heavy tailed distributions. This motivates us to further study this question under a wide range of loads and job size variabilities in the context of FCFS before studying priority systems.