|
Figure 7 shows how per-node bandwidth varies as we increase the number of nodes in Colyseus, where each additional node has 4 players + 4 monsters in the edge-server case (a) and 1 player in the peer-to-peer case (b). Each point is calculated as the average of the mean bandwidth used by each node in the system over a 5 minute period of the game. Since game traffic is fairly bursty (see Figure 9), we also plot the 95th percentile of 1 second burst rates as a thin error bar over each point. We represent load balance with the thick error bars indicate 1 standard deviation from the mean (across servers). Recall that our existing implementation does not yet perform dynamic load balancing. We revisit this issue in Section 7.5.
For comparison, we plot the bandwidth used by the single server of a client-server architecture and the average bandwidth used by all nodes in a broadcast architecture. We no not include bandwidth consumed by communication to clients in the Colyseys or broadcast cases since we assume that the servers are colocated on the same machine or close to the clients they are serving. If we had included client server costs, then each server in the edge-server case would only take on an additional 200-220kbps and 10-30kbps in the peer-to-peer case.
In the edge-server scenario (Figure 7(a)), Colyseus's bandwidth usage is a factor of 3-4 less than the broadcast architecture and each node sends a factor of 10 less than the single centralized server. Nonetheless, the aggregate bandwidth used by all Colyseus nodes combined is almost a factor of 10 times more than the total cost incured by a single server node. In the edge-server scenario, there are 8 objects per server (4 players + 4 monsters) that introduce subscriptions, and from Figure 6(a) we see that each subscription incompasses about 1-2% of the entire game world in our workload. Hence, since objects are essentially placed randomly, each node is always interested in 8-16% or more of the other servers at any given time. Thus, there is a lot of overhead for object interaction in this scenario. However, we note that despite this, the scaling behavior is similar to the central server architecture, so additional players can be accomodated with approximately the same amount of additional distributed resources.
In the peer-to-peer scenario (Figure 7(b)), Colyseus performs much better relative to the alternative architectures, achieving over an order-of-magnitude improvement over either. Since each node only hosts a single subscribing object (the 1 player), each node is required to know about less than in the edge-server case. Nontheless, there is still overhead, as Colyseus uses 5-6 times more bandwidth in aggregate than the client-server scenario. What is more interesting perhaps is that even with 150 players, we are able to support the game with less than 110kbps on average and less than 300kbps in almost all time instances. Keeping in mind that our messages are unoptimized and delta sizes are likely to be smaller for optimized games, this suggests that medium sized peer-to-peer deployment scenarios can be practical with today's broadband technologies.